2021-03-09T02:42:48.496 INFO:root:teuthology version: 1.1.0-3082387b 2021-03-09T02:42:48.498 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2021-03-08_21:10:20-rbd-wip-yuri5-testing-2021-03-08-1049-pacific-distro-basic-smithi --verbose --description rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2021-03-08_21:10:20-rbd-wip-yuri5-testing-2021-03-08-1049-pacific-distro-basic-smithi/5947743 -- /home/teuthworker/archive/yuriw-2021-03-08_21:10:20-rbd-wip-yuri5-testing-2021-03-08-1049-pacific-distro-basic-smithi/5947743/orig.config.yaml 2021-03-09T02:42:48.570 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-03-09T02:42:48.630 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2021-03-08_21:10:20-rbd-wip-yuri5-testing-2021-03-08-1049-pacific-distro-basic-smithi/5947743 branch: wip-yuri5-testing-2021-03-08-1049-pacific description: rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock} email: ceph-qa@ceph.io first_in_suite: false job_id: '5947743' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: run two ceph clusters and install rbd-mirror - desc: configure the permissions for client.mirror - desc: 2 ceph clusters with 1 mon and 3 osds each - desc: run four rbd-mirror daemons per cluster - desc: run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon name: yuriw-2021-03-08_21:10:20-rbd-wip-yuri5-testing-2021-03-08-1049-pacific-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 3 size: 30 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: wip-yuri5-testing-2021-03-08-1049-pacific ceph: conf: client: debug rbd: 20 debug rbd_mirror: 15 log to stderr: false rbd default features: 125 rbd mirror image policy type: simple client.mirror.0: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.1: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.2: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid client.mirror.3: admin socket: /var/run/ceph/rbd-mirror.$cluster-$name.asok pid file: /var/run/ceph/rbd-mirror.$cluster-$name.pid global: mon client directed command retry: 5 ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 9e448db9e385343da144613d840349182443aeee ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: 9e448db9e385343da144613d840349182443aeee thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri5-testing-2021-03-08-1049-pacific sha1: 9e448db9e385343da144613d840349182443aeee owner: scheduled_yuriw@teuthology priority: 80 repo: https://github.com/ceph/ceph-ci.git roles: - - cluster1.mon.a - cluster1.mgr.x - cluster2.mgr.x - cluster1.osd.0 - cluster1.osd.1 - cluster1.osd.2 - cluster1.client.0 - cluster2.client.0 - - cluster2.mon.a - cluster2.osd.0 - cluster2.osd.1 - cluster2.osd.2 - cluster1.client.mirror - cluster1.client.mirror.0 - cluster1.client.mirror.1 - cluster1.client.mirror.2 - cluster1.client.mirror.3 - cluster1.client.mirror.4 - cluster1.client.mirror.5 - cluster1.client.mirror.6 - cluster2.client.mirror - cluster2.client.mirror.0 - cluster2.client.mirror.1 - cluster2.client.mirror.2 - cluster2.client.mirror.3 - cluster2.client.mirror.4 - cluster2.client.mirror.5 - cluster2.client.mirror.6 sha1: 9e448db9e385343da144613d840349182443aeee sleep_before_teardown: 0 suite: rbd suite_branch: wip-yuri5-testing-2021-03-08-1049-pacific suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_9e448db9e385343da144613d840349182443aeee/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 9e448db9e385343da144613d840349182443aeee targets: smithi092.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGM9Wq+tpjME44vX4r5HVAYDKQHOPpKMuKkIN+sN4bXOXrL38m4IlDpX7+u11H/YKmLYO9J8qq8LeIhaPVveNNtI00S6NRcL2gCFTKBc3cGAuajI1w7T5UcmQcjTDdaC6irQDmclSzc/LrhabDa/E8LWZa/J4pJf2j93rHf5ALtl6DxhOPtDMtksmxnIlvXWznZj0nTqZXZjx2V3DJ8edC0ENLQTQuG/GIUX4uAXK5T40vzv21g1EG4jAiUKZXB1t91+utaws6miPeVZ6W94mAkMdmqdlkcbccy4PWRXiz4+MNFNhUtNBxtSgeuFc8gQfrZXKWdIxyHpSoybONFFN/ smithi144.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQClEsHrqwB5WWdjBzHc5HfVxIvP9a9tTOK/FXO13yldAlBA6wEnj8ybBtfnvsWbhjdORYKBxUEiPAcNjeDd6CdsyLAIAI/Yo6n//ZuBnBEURpcBxLmeyQf8YLvKLJ8eUYJTdiFMUiimvqszh7S1Uzqi5crdAP9jWyZjL7Bo+7itY52pzDWhxFtARQfcV/FOUBQymt8GK7We3s1Dlc/vfHw0JTaxeUfwCBOx2L2GxeGAGLpnnJhV6ohsFhIb7ZnfMRAR2PV4osf84S67vArQj7H5H/XqQJaSVc5EUkPodGwiRxqplJCgtlpb205SiEMOBwB/XX5ppSSCp0RfBj91NwdX tasks: - install: extra_packages: - rbd-mirror - ceph: cluster: cluster1 - ceph: cluster: cluster2 - exec: cluster1.client.mirror.0: - sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd' - sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd' - rbd-mirror: client: cluster1.client.mirror.0 thrash: true - rbd-mirror: client: cluster1.client.mirror.1 thrash: true - rbd-mirror: client: cluster1.client.mirror.2 thrash: true - rbd-mirror: client: cluster1.client.mirror.3 thrash: true - rbd-mirror: client: cluster2.client.mirror.0 thrash: true - rbd-mirror: client: cluster2.client.mirror.1 thrash: true - rbd-mirror: client: cluster2.client.mirror.2 thrash: true - rbd-mirror: client: cluster2.client.mirror.3 thrash: true - rbd-mirror-thrash: cluster: cluster1 - rbd-mirror-thrash: cluster: cluster2 - workunit: clients: cluster1.client.mirror: - rbd/rbd_mirror_stress.sh env: CEPH_ARGS: '' MIRROR_IMAGE_MODE: snapshot MIRROR_POOL_MODE: image RBD_IMAGE_FEATURES: layering,exclusive-lock RBD_MIRROR_INSTANCES: '4' RBD_MIRROR_USE_EXISTING_CLUSTER: '1' RBD_MIRROR_USE_RBD_MIRROR: '1' timeout: 6h teuthology_branch: master teuthology_sha1: 3082387bba74fcd24c9700593d10418152d53c97 timestamp: 2021-03-08_21:10:20 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.25973 2021-03-09T02:42:48.631 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_9e448db9e385343da144613d840349182443aeee/qa; will attempt to use it 2021-03-09T02:42:48.632 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_9e448db9e385343da144613d840349182443aeee/qa/tasks 2021-03-09T02:42:48.632 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-03-09T02:42:48.634 INFO:teuthology.task.internal:Checking packages... 2021-03-09T02:42:48.655 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash '9e448db9e385343da144613d840349182443aeee' 2021-03-09T02:42:48.656 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-03-09T02:42:48.656 INFO:teuthology.packaging:ref: None 2021-03-09T02:42:48.656 INFO:teuthology.packaging:tag: None 2021-03-09T02:42:48.657 INFO:teuthology.packaging:branch: wip-yuri5-testing-2021-03-08-1049-pacific 2021-03-09T02:42:48.657 INFO:teuthology.packaging:sha1: 9e448db9e385343da144613d840349182443aeee 2021-03-09T02:42:48.658 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-yuri5-testing-2021-03-08-1049-pacific 2021-03-09T02:42:48.820 INFO:teuthology.task.internal:Found packages for ceph version 16.1.0-627-g9e448db9-1bionic 2021-03-09T02:42:48.821 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-03-09T02:42:48.854 INFO:teuthology.task.internal:no buildpackages task found 2021-03-09T02:42:48.855 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-03-09T02:42:48.876 INFO:teuthology.task.internal:Saving configuration 2021-03-09T02:42:48.896 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-03-09T02:42:48.906 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-03-09T02:42:48.925 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-03-09 02:37:24.845610', 'description': '/home/teuthworker/archive/yuriw-2021-03-08_21:10:20-rbd-wip-yuri5-testing-2021-03-08-1049-pacific-distro-basic-smithi/5947743', 'locked_by': 'scheduled_yuriw@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMEIPXBFaYBkJmXNWVVdU049+BYwI4vdaxbP8MarIINo', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi092.front.sepia.ceph.com'} 2021-03-09T02:42:48.947 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2021-03-09 02:37:24.845591', 'description': '/home/teuthworker/archive/yuriw-2021-03-08_21:10:20-rbd-wip-yuri5-testing-2021-03-08-1049-pacific-distro-basic-smithi/5947743', 'locked_by': 'scheduled_yuriw@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFAoT50HMAq5r/3v6I3oLaNMxmnGvQs2vA1gLmdwkIBS', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi144.front.sepia.ceph.com'} 2021-03-09T02:42:48.947 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-03-09T02:42:48.964 INFO:teuthology.task.internal:roles: ubuntu@smithi092.front.sepia.ceph.com - ['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'] 2021-03-09T02:42:48.965 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6'] 2021-03-09T02:42:48.965 INFO:teuthology.run_tasks:Running task console_log... 2021-03-09T02:42:49.040 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f07b13f28c8>, signals=[15]) 2021-03-09T02:42:49.041 INFO:teuthology.run_tasks:Running task internal.connect... 2021-03-09T02:42:49.048 INFO:teuthology.task.internal:Opening connections... 2021-03-09T02:42:49.049 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi092.front.sepia.ceph.com 2021-03-09T02:42:49.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-09T02:42:49.124 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2021-03-09T02:42:49.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-09T02:42:49.201 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-03-09T02:42:49.210 DEBUG:teuthology.orchestra.run.smithi092:> uname -m 2021-03-09T02:42:49.329 INFO:teuthology.orchestra.run.smithi092.stdout:x86_64 2021-03-09T02:42:49.329 DEBUG:teuthology.orchestra.run.smithi092:> cat /etc/os-release 2021-03-09T02:42:49.377 INFO:teuthology.orchestra.run.smithi092.stdout:NAME="Ubuntu" 2021-03-09T02:42:49.378 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-03-09T02:42:49.378 INFO:teuthology.orchestra.run.smithi092.stdout:ID=ubuntu 2021-03-09T02:42:49.378 INFO:teuthology.orchestra.run.smithi092.stdout:ID_LIKE=debian 2021-03-09T02:42:49.379 INFO:teuthology.orchestra.run.smithi092.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-03-09T02:42:49.379 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_ID="18.04" 2021-03-09T02:42:49.380 INFO:teuthology.orchestra.run.smithi092.stdout:HOME_URL="https://www.ubuntu.com/" 2021-03-09T02:42:49.380 INFO:teuthology.orchestra.run.smithi092.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-03-09T02:42:49.380 INFO:teuthology.orchestra.run.smithi092.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-03-09T02:42:49.380 INFO:teuthology.orchestra.run.smithi092.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-03-09T02:42:49.381 INFO:teuthology.orchestra.run.smithi092.stdout:VERSION_CODENAME=bionic 2021-03-09T02:42:49.381 INFO:teuthology.orchestra.run.smithi092.stdout:UBUNTU_CODENAME=bionic 2021-03-09T02:42:49.382 INFO:teuthology.lock.ops:Updating smithi092.front.sepia.ceph.com on lock server 2021-03-09T02:42:49.398 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2021-03-09T02:42:49.404 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2021-03-09T02:42:49.405 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2021-03-09T02:42:49.454 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="Ubuntu" 2021-03-09T02:42:49.455 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2021-03-09T02:42:49.455 INFO:teuthology.orchestra.run.smithi144.stdout:ID=ubuntu 2021-03-09T02:42:49.456 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE=debian 2021-03-09T02:42:49.456 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2021-03-09T02:42:49.457 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="18.04" 2021-03-09T02:42:49.457 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://www.ubuntu.com/" 2021-03-09T02:42:49.457 INFO:teuthology.orchestra.run.smithi144.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2021-03-09T02:42:49.457 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2021-03-09T02:42:49.457 INFO:teuthology.orchestra.run.smithi144.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2021-03-09T02:42:49.458 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_CODENAME=bionic 2021-03-09T02:42:49.458 INFO:teuthology.orchestra.run.smithi144.stdout:UBUNTU_CODENAME=bionic 2021-03-09T02:42:49.458 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2021-03-09T02:42:49.473 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-03-09T02:42:49.518 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-03-09T02:42:49.537 INFO:teuthology.task.internal:Checking for old test directory... 2021-03-09T02:42:49.538 DEBUG:teuthology.orchestra.run.smithi092:> test '!' -e /home/ubuntu/cephtest 2021-03-09T02:42:49.541 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2021-03-09T02:42:49.546 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-03-09T02:42:49.555 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-03-09T02:42:49.555 DEBUG:teuthology.orchestra.run.smithi092:> test -z $(ls -A /var/lib/ceph) 2021-03-09T02:42:49.590 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2021-03-09T02:42:49.605 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-03-09T02:42:49.647 INFO:teuthology.run_tasks:Running task kernel... 2021-03-09T02:42:49.664 DEBUG:teuthology.task.kernel:normalized config {'cluster1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster1.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mgr.x': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.0': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.1': {'kdb': True, 'sha1': 'distro'}, 'cluster1.osd.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.1': {'kdb': True, 'sha1': 'distro'}, 'cluster2.osd.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.0': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.1': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.2': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.3': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.4': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.5': {'kdb': True, 'sha1': 'distro'}, 'cluster1.client.mirror.6': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.0': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.1': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.2': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.3': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.4': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.5': {'kdb': True, 'sha1': 'distro'}, 'cluster2.client.mirror.6': {'kdb': True, 'sha1': 'distro'}} 2021-03-09T02:42:49.666 INFO:teuthology.task.kernel:config {'cluster1.mon.a': {'kdb': True, 'sha1': 'distro'}, 'cluster2.mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-03-09T02:42:49.666 DEBUG:teuthology.orchestra.run.smithi092:> uname -r 2021-03-09T02:42:49.673 INFO:teuthology.orchestra.run.smithi092.stdout:4.15.0-136-generic 2021-03-09T02:42:49.673 INFO:teuthology.task.kernel:Running kernel on smithi092: 4.15.0-136-generic 2021-03-09T02:42:49.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get clean 2021-03-09T02:42:49.780 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get update 2021-03-09T02:42:49.898 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-03-09T02:42:50.023 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-03-09T02:42:50.103 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-03-09T02:42:50.148 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,573 kB] 2021-03-09T02:42:50.253 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [926 kB] 2021-03-09T02:42:50.270 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [302 kB] 2021-03-09T02:42:50.272 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [17.7 kB] 2021-03-09T02:42:50.272 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [245 kB] 2021-03-09T02:42:50.274 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [31.7 kB] 2021-03-09T02:42:50.275 INFO:teuthology.orchestra.run.smithi092.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,113 kB] 2021-03-09T02:42:50.285 INFO:teuthology.orchestra.run.smithi092.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [979 kB] 2021-03-09T02:42:50.295 INFO:teuthology.orchestra.run.smithi092.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [249 kB] 2021-03-09T02:42:50.297 INFO:teuthology.orchestra.run.smithi092.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,480 B] 2021-03-09T02:42:50.297 INFO:teuthology.orchestra.run.smithi092.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.1 kB] 2021-03-09T02:42:50.298 INFO:teuthology.orchestra.run.smithi092.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [4,412 B] 2021-03-09T02:42:50.373 INFO:teuthology.orchestra.run.smithi092.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-03-09T02:42:50.488 INFO:teuthology.orchestra.run.smithi092.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,913 kB] 2021-03-09T02:42:50.817 INFO:teuthology.orchestra.run.smithi092.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,228 kB] 2021-03-09T02:42:50.871 INFO:teuthology.orchestra.run.smithi092.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [266 kB] 2021-03-09T02:42:50.880 INFO:teuthology.orchestra.run.smithi092.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [24.3 kB] 2021-03-09T02:42:50.881 INFO:teuthology.orchestra.run.smithi092.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [35.3 kB] 2021-03-09T02:42:50.882 INFO:teuthology.orchestra.run.smithi092.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,719 kB] 2021-03-09T02:42:50.925 INFO:teuthology.orchestra.run.smithi092.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,561 kB] 2021-03-09T02:42:50.979 INFO:teuthology.orchestra.run.smithi092.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [24.9 kB] 2021-03-09T02:42:50.980 INFO:teuthology.orchestra.run.smithi092.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.6 kB] 2021-03-09T02:42:50.980 INFO:teuthology.orchestra.run.smithi092.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,464 B] 2021-03-09T02:42:51.896 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 12.5 MB in 2s (6,545 kB/s) 2021-03-09T02:42:53.349 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2021-03-09T02:42:53.395 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-03-09T02:42:53.472 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2021-03-09T02:42:53.731 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2021-03-09T02:42:53.732 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2021-03-09T02:42:53.812 INFO:teuthology.orchestra.run.smithi092.stderr:E: Unable to locate package linux-image-current-generic 2021-03-09T02:42:53.813 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-03-09T02:42:53.814 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-03-09T02:42:53.890 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2021-03-09T02:42:54.119 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2021-03-09T02:42:54.119 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2021-03-09T02:42:54.323 INFO:teuthology.orchestra.run.smithi092.stdout:linux-image-generic is already the newest version (4.15.0.136.123). 2021-03-09T02:42:54.323 INFO:teuthology.orchestra.run.smithi092.stdout:linux-image-generic set to manually installed. 2021-03-09T02:42:54.323 INFO:teuthology.orchestra.run.smithi092.stdout:0 upgraded, 0 newly installed, 0 to remove and 15 not upgraded. 2021-03-09T02:42:54.324 DEBUG:teuthology.orchestra.run.smithi092:> dpkg -s linux-image-generic 2021-03-09T02:42:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:Package: linux-image-generic 2021-03-09T02:42:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:Status: install ok installed 2021-03-09T02:42:54.346 INFO:teuthology.orchestra.run.smithi092.stdout:Priority: optional 2021-03-09T02:42:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:Section: kernel 2021-03-09T02:42:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:Installed-Size: 17 2021-03-09T02:42:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:Maintainer: Ubuntu Kernel Team 2021-03-09T02:42:54.347 INFO:teuthology.orchestra.run.smithi092.stdout:Architecture: amd64 2021-03-09T02:42:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:Source: linux-meta 2021-03-09T02:42:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:Version: 4.15.0.136.123 2021-03-09T02:42:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-03-09T02:42:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:Depends: linux-image-4.15.0-136-generic, linux-modules-extra-4.15.0-136-generic, linux-firmware, intel-microcode, amd64-microcode 2021-03-09T02:42:54.348 INFO:teuthology.orchestra.run.smithi092.stdout:Recommends: thermald 2021-03-09T02:42:54.349 INFO:teuthology.orchestra.run.smithi092.stdout:Description: Generic Linux kernel image 2021-03-09T02:42:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: This package will always depend on the latest generic kernel image 2021-03-09T02:42:54.349 INFO:teuthology.orchestra.run.smithi092.stdout: available. 2021-03-09T02:42:54.349 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2021-03-09T02:42:54.350 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2021-03-09T02:42:54.357 INFO:teuthology.orchestra.run.smithi144.stdout:4.15.0-136-generic 2021-03-09T02:42:54.357 INFO:teuthology.task.kernel:Running kernel on smithi144: 4.15.0-136-generic 2021-03-09T02:42:54.359 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get clean 2021-03-09T02:42:54.471 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2021-03-09T02:42:54.746 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-03-09T02:42:54.747 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2021-03-09T02:42:54.811 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2021-03-09T02:42:55.069 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2021-03-09T02:42:55.195 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [1,573 kB] 2021-03-09T02:42:55.319 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,228 kB] 2021-03-09T02:42:55.617 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [926 kB] 2021-03-09T02:42:55.640 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [302 kB] 2021-03-09T02:42:55.646 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [245 kB] 2021-03-09T02:42:55.652 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [17.7 kB] 2021-03-09T02:42:55.653 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [31.7 kB] 2021-03-09T02:42:55.654 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,113 kB] 2021-03-09T02:42:55.687 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1,913 kB] 2021-03-09T02:42:55.723 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [979 kB] 2021-03-09T02:42:55.746 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [249 kB] 2021-03-09T02:42:55.754 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,480 B] 2021-03-09T02:42:55.754 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.1 kB] 2021-03-09T02:42:55.754 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [4,412 B] 2021-03-09T02:42:55.790 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [266 kB] 2021-03-09T02:42:55.801 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [24.3 kB] 2021-03-09T02:42:55.802 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [35.3 kB] 2021-03-09T02:42:55.803 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,561 kB] 2021-03-09T02:42:55.852 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,719 kB] 2021-03-09T02:42:55.888 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [24.9 kB] 2021-03-09T02:42:55.889 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.6 kB] 2021-03-09T02:42:55.890 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,464 B] 2021-03-09T02:42:57.177 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 12.5 MB in 2s (5,024 kB/s) 2021-03-09T02:42:58.592 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2021-03-09T02:42:58.640 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-current-generic 2021-03-09T02:42:58.719 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2021-03-09T02:42:58.980 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2021-03-09T02:42:58.980 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2021-03-09T02:42:59.066 INFO:teuthology.orchestra.run.smithi144.stderr:E: Unable to locate package linux-image-current-generic 2021-03-09T02:42:59.067 DEBUG:teuthology.orchestra.run:got remote process result: 100 2021-03-09T02:42:59.068 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2021-03-09T02:42:59.150 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2021-03-09T02:42:59.400 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2021-03-09T02:42:59.400 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2021-03-09T02:42:59.606 INFO:teuthology.orchestra.run.smithi144.stdout:linux-image-generic is already the newest version (4.15.0.136.123). 2021-03-09T02:42:59.606 INFO:teuthology.orchestra.run.smithi144.stdout:linux-image-generic set to manually installed. 2021-03-09T02:42:59.606 INFO:teuthology.orchestra.run.smithi144.stdout:0 upgraded, 0 newly installed, 0 to remove and 15 not upgraded. 2021-03-09T02:42:59.607 DEBUG:teuthology.orchestra.run.smithi144:> dpkg -s linux-image-generic 2021-03-09T02:42:59.623 INFO:teuthology.orchestra.run.smithi144.stdout:Package: linux-image-generic 2021-03-09T02:42:59.624 INFO:teuthology.orchestra.run.smithi144.stdout:Status: install ok installed 2021-03-09T02:42:59.624 INFO:teuthology.orchestra.run.smithi144.stdout:Priority: optional 2021-03-09T02:42:59.625 INFO:teuthology.orchestra.run.smithi144.stdout:Section: kernel 2021-03-09T02:42:59.625 INFO:teuthology.orchestra.run.smithi144.stdout:Installed-Size: 17 2021-03-09T02:42:59.625 INFO:teuthology.orchestra.run.smithi144.stdout:Maintainer: Ubuntu Kernel Team 2021-03-09T02:42:59.625 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: amd64 2021-03-09T02:42:59.626 INFO:teuthology.orchestra.run.smithi144.stdout:Source: linux-meta 2021-03-09T02:42:59.626 INFO:teuthology.orchestra.run.smithi144.stdout:Version: 4.15.0.136.123 2021-03-09T02:42:59.626 INFO:teuthology.orchestra.run.smithi144.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.1), zfs-modules (= 0.7.5-1ubuntu16.10) 2021-03-09T02:42:59.626 INFO:teuthology.orchestra.run.smithi144.stdout:Depends: linux-image-4.15.0-136-generic, linux-modules-extra-4.15.0-136-generic, linux-firmware, intel-microcode, amd64-microcode 2021-03-09T02:42:59.626 INFO:teuthology.orchestra.run.smithi144.stdout:Recommends: thermald 2021-03-09T02:42:59.627 INFO:teuthology.orchestra.run.smithi144.stdout:Description: Generic Linux kernel image 2021-03-09T02:42:59.627 INFO:teuthology.orchestra.run.smithi144.stdout: This package will always depend on the latest generic kernel image 2021-03-09T02:42:59.627 INFO:teuthology.orchestra.run.smithi144.stdout: available. 2021-03-09T02:42:59.627 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2021-03-09T02:42:59.628 INFO:teuthology.task.kernel:Enabling kdb on cluster1.mon.a... 2021-03-09T02:42:59.628 DEBUG:teuthology.orchestra.run.smithi092:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-03-09T02:42:59.642 INFO:teuthology.orchestra.run.smithi092.stdout:ttyS1 2021-03-09T02:42:59.678 INFO:teuthology.task.kernel:Enabling kdb on cluster2.mon.a... 2021-03-09T02:42:59.678 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-03-09T02:42:59.698 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2021-03-09T02:42:59.718 INFO:teuthology.run_tasks:Running task internal.base... 2021-03-09T02:42:59.730 INFO:teuthology.task.internal:Creating test directory... 2021-03-09T02:42:59.731 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-03-09T02:42:59.734 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-03-09T02:42:59.766 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-03-09T02:42:59.777 INFO:teuthology.run_tasks:Running task internal.archive... 2021-03-09T02:42:59.796 INFO:teuthology.task.internal:Creating archive directory... 2021-03-09T02:42:59.797 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-03-09T02:42:59.801 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-03-09T02:42:59.832 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-03-09T02:42:59.841 INFO:teuthology.task.internal:Enabling coredump saving... 2021-03-09T02:42:59.842 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-03-09T02:42:59.861 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-03-09T02:42:59.886 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-09T02:42:59.890 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-09T02:42:59.893 INFO:teuthology.orchestra.run.smithi092.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-09T02:42:59.896 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-03-09T02:42:59.897 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-03-09T02:42:59.904 INFO:teuthology.task.internal:Configuring sudo... 2021-03-09T02:42:59.905 DEBUG:teuthology.orchestra.run.smithi092:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-03-09T02:42:59.937 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-03-09T02:42:59.953 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-03-09T02:42:59.962 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-03-09T02:42:59.963 DEBUG:teuthology.orchestra.run.smithi092:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-03-09T02:42:59.994 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-03-09T02:43:00.006 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-03-09T02:43:00.046 DEBUG:teuthology.orchestra.run.smithi092:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-03-09T02:43:00.094 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:43:00.095 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-03-09T02:43:00.157 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-03-09T02:43:00.164 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-03-09T02:43:00.215 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:43:00.215 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-03-09T02:43:00.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo service rsyslog restart 2021-03-09T02:43:00.292 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2021-03-09T02:43:00.373 INFO:teuthology.run_tasks:Running task internal.timer... 2021-03-09T02:43:00.382 INFO:teuthology.task.internal:Starting timer... 2021-03-09T02:43:00.382 INFO:teuthology.run_tasks:Running task pcp... 2021-03-09T02:43:00.420 INFO:teuthology.run_tasks:Running task selinux... 2021-03-09T02:43:00.436 DEBUG:teuthology.task.selinux:Excluding smithi092: OS 'ubuntu' does not support SELinux 2021-03-09T02:43:00.437 DEBUG:teuthology.task.selinux:Excluding smithi144: OS 'ubuntu' does not support SELinux 2021-03-09T02:43:00.437 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-03-09T02:43:00.437 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2021-03-09T02:43:00.438 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-03-09T02:43:00.438 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-03-09T02:43:00.448 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated or references a specific commit; assuming it is current 2021-03-09T02:43:00.449 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to origin/master 2021-03-09T02:43:00.466 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2021-03-09T02:43:00.468 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi092.front.sepia.ceph.com,smithi144.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-03-09T02:43:00.837 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-03-09T02:43:01.717 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-03-09T02:43:03.925 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-03-09T02:43:03.947 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:03 +0000 (0:00:00.037) 0:00:00.037 ********* 2021-03-09T02:43:03.957 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:03 +0000 (0:00:00.010) 0:00:00.047 ********* 2021-03-09T02:43:05.556 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* [DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host 2021-03-09T02:43:05.558 INFO:teuthology.task.ansible.out: smithi092.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [smithi092.front.sepia.ceph.com] 2021-03-09T02:43:05.592 INFO:teuthology.task.ansible.out:[DEPRECATION WARNING]: Distribution Ubuntu 18.04 on host smithi144.front.sepia.ceph.com should use /usr/bin/python3, but is using /usr/bin/python for backward compatibility with prior Ansible releases. A future Ansible release will default to using the discovered platform python for this host. See https://docs.ansible.com/ansible/2.9/reference_appendices/inter preter_discovery.html for more information. This feature will be removed in version 2.12. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg. ok: [smithi144.front.sepia.ceph.com] 2021-03-09T02:43:05.621 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:05 +0000 (0:00:01.664) 0:00:01.711 ********* 2021-03-09T02:43:05.664 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:05 +0000 (0:00:00.042) 0:00:01.754 ********* 2021-03-09T02:43:05.900 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-03-09T02:43:05.902 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-03-09T02:43:05.924 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:05 +0000 (0:00:00.260) 0:00:02.014 ********* 2021-03-09T02:43:05.943 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:05 +0000 (0:00:00.018) 0:00:02.033 ********* 2021-03-09T02:43:06.546 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-03-09T02:43:06.548 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} changed: [smithi144.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-03-09T02:43:06.570 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:06 +0000 (0:00:00.627) 0:00:02.661 ********* 2021-03-09T02:43:06.596 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:06 +0000 (0:00:00.025) 0:00:02.686 ********* 2021-03-09T02:43:06.960 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-03-09T02:43:06.962 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.083196", "end": "2021-03-09 02:43:06.480770", "rc": 0, "start": "2021-03-09 02:43:06.397574", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.081080", "end": "2021-03-09 02:43:06.694090", "rc": 0, "start": "2021-03-09 02:43:06.613010", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2021-03-09T02:43:06.986 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:06 +0000 (0:00:00.389) 0:00:03.076 ********* 2021-03-09T02:43:07.006 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:06 +0000 (0:00:00.019) 0:00:03.096 ********* 2021-03-09T02:43:07.253 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-03-09T02:43:07.255 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} ok: [smithi144.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-09T02:43:07.278 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:07 +0000 (0:00:00.272) 0:00:03.369 ********* 2021-03-09T02:43:07.299 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:07 +0000 (0:00:00.020) 0:00:03.389 ********* 2021-03-09T02:43:08.185 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2021-03-09T02:43:08.187 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615257787.3176398-32573-30919476224861/source", "state": "file", "uid": 0} 2021-03-09T02:43:08.212 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:08 +0000 (0:00:00.913) 0:00:04.302 ********* 2021-03-09T02:43:08.219 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615257787.341635-32576-215104709506038/source", "state": "file", "uid": 0} 2021-03-09T02:43:08.245 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:08 +0000 (0:00:00.033) 0:00:04.336 ********* 2021-03-09T02:43:08.702 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-03-09T02:43:08.704 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-03-09T02:43:08.706 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail 2021-03-09T02:43:08.708 INFO:teuthology.task.ansible.out:.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@ 2021-03-09T02:43:08.709 INFO:teuthology.task.ansible.out:aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-03-09T02:43:08.793 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-03-09T02:43:08.796 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-03-09T02:43:08.952 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:08 +0000 (0:00:00.706) 0:00:05.042 ********* 2021-03-09T02:43:08.974 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:08 +0000 (0:00:00.021) 0:00:05.064 ********* 2021-03-09T02:43:09.675 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi092.front.sepia.ceph.com] 2021-03-09T02:43:09.713 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] 2021-03-09T02:43:09.753 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. Tuesday 09 March 2021 02:43:09 +0000 (0:00:00.778) 0:00:05.843 ********* 2021-03-09T02:43:09.820 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:09 +0000 (0:00:00.066) 0:00:05.910 ********* 2021-03-09T02:43:10.180 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-03-09T02:43:10.182 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.157530", "end": "2021-03-09 02:43:10.160213", "rc": 0, "start": "2021-03-09 02:43:10.002683", "stderr": "", "stderr_lines": [], "stdout": "c31004c4720a55ac12a95d66c32e64f592877604", "stdout_lines": ["c31004c4720a55ac12a95d66c32e64f592877604"]} 2021-03-09T02:43:10.206 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.385) 0:00:06.295 ********* 2021-03-09T02:43:10.267 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.193100", "end": "2021-03-09 02:43:10.230803", "rc": 0, "start": "2021-03-09 02:43:10.037703", "stderr": "", "stderr_lines": [], "stdout": "c31004c4720a55ac12a95d66c32e64f592877604", "stdout_lines": ["c31004c4720a55ac12a95d66c32e64f592877604"]} 2021-03-09T02:43:10.294 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.088) 0:00:06.384 ********* 2021-03-09T02:43:10.343 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-03-09T02:43:10.345 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.008812", "end": "2021-03-09 02:43:09.872092", "failed_when_result": false, "rc": 0, "start": "2021-03-09 02:43:09.863280", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-03-09T02:43:10.375 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.080) 0:00:06.465 ********* 2021-03-09T02:43:10.445 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** 2021-03-09T02:43:10.447 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:10.474 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.098) 0:00:06.564 ********* 2021-03-09T02:43:10.480 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-03-09T02:43:10.482 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.021395", "end": "2021-03-09 02:43:10.184867", "failed_when_result": false, "rc": 0, "start": "2021-03-09 02:43:10.163472", "stderr": "", "stderr_lines": [], "stdout": "df8946f08e870e2d2565b79099289ca48bd825f3", "stdout_lines": ["df8946f08e870e2d2565b79099289ca48bd825f3"]} 2021-03-09T02:43:10.508 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.034) 0:00:06.598 ********* 2021-03-09T02:43:10.545 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-03-09T02:43:10.548 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:10.571 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.063) 0:00:06.661 ********* 2021-03-09T02:43:10.587 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** 2021-03-09T02:43:10.589 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:10.614 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.042) 0:00:06.704 ********* 2021-03-09T02:43:10.631 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** 2021-03-09T02:43:10.632 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:10.663 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.048) 0:00:06.753 ********* 2021-03-09T02:43:10.690 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-03-09T02:43:10.692 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:10.715 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.052) 0:00:06.805 ********* 2021-03-09T02:43:10.743 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-03-09T02:43:10.746 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:10.771 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.055) 0:00:06.861 ********* 2021-03-09T02:43:10.773 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:10.790 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** 2021-03-09T02:43:10.791 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:10.816 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.044) 0:00:06.906 ********* 2021-03-09T02:43:10.853 INFO:teuthology.task.ansible.out: TASK [Filter the managed_admin_users list] ************************************* 2021-03-09T02:43:10.855 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:10.877 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.060) 0:00:06.967 ********* 2021-03-09T02:43:10.894 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-03-09T02:43:10.895 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:10.919 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:10 +0000 (0:00:00.042) 0:00:07.009 ********* 2021-03-09T02:43:11.000 INFO:teuthology.task.ansible.out: TASK [Filter the managed_admin_users list] ************************************* 2021-03-09T02:43:11.002 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:11.023 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:11 +0000 (0:00:00.103) 0:00:07.113 ********* 2021-03-09T02:43:11.073 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-03-09T02:43:11.075 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-09T02:43:11.108 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:11 +0000 (0:00:00.085) 0:00:07.198 ********* 2021-03-09T02:43:11.221 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-09T02:43:11.245 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:11 +0000 (0:00:00.136) 0:00:07.335 ********* 2021-03-09T02:43:11.368 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-09T02:43:11.369 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-03-09T02:43:11.494 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-03-09T02:43:11.496 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:11.507 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-03-09T02:43:11.656 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-03-09T02:43:11.657 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-03-09T02:43:11.794 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-03-09T02:43:11.806 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-03-09T02:43:11.955 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-03-09T02:43:11.956 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-03-09T02:43:12.094 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-03-09T02:43:12.116 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-03-09T02:43:12.241 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-03-09T02:43:12.242 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:12.254 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-03-09T02:43:12.391 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-03-09T02:43:12.393 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:12.406 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-03-09T02:43:12.546 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-03-09T02:43:12.568 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-03-09T02:43:12.727 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-03-09T02:43:12.740 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-03-09T02:43:12.868 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-03-09T02:43:12.901 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-03-09T02:43:13.007 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-03-09T02:43:13.052 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-03-09T02:43:13.160 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-03-09T02:43:13.193 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-03-09T02:43:13.310 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-03-09T02:43:13.346 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-03-09T02:43:13.471 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-03-09T02:43:13.524 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-03-09T02:43:13.526 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:13.622 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-03-09T02:43:13.666 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-03-09T02:43:13.772 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-03-09T02:43:13.816 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-03-09T02:43:13.912 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-03-09T02:43:13.965 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-03-09T02:43:14.060 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-03-09T02:43:14.126 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1118} 2021-03-09T02:43:14.202 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-03-09T02:43:14.276 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-03-09T02:43:14.362 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-03-09T02:43:14.415 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-03-09T02:43:14.500 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-03-09T02:43:14.580 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1121} 2021-03-09T02:43:14.698 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-03-09T02:43:14.762 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-03-09T02:43:14.848 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-03-09T02:43:14.920 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-03-09T02:43:15.010 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-03-09T02:43:15.077 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-03-09T02:43:15.159 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-03-09T02:43:15.234 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-03-09T02:43:15.308 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-03-09T02:43:15.384 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-03-09T02:43:15.461 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-03-09T02:43:15.524 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-03-09T02:43:15.598 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-03-09T02:43:15.665 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-03-09T02:43:15.739 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-03-09T02:43:15.825 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-03-09T02:43:15.899 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-03-09T02:43:15.967 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-03-09T02:43:16.081 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-03-09T02:43:16.135 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-03-09T02:43:16.231 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-03-09T02:43:16.277 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-03-09T02:43:16.375 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-03-09T02:43:16.430 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1133} 2021-03-09T02:43:16.432 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:16.515 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-03-09T02:43:16.582 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-03-09T02:43:16.656 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-03-09T02:43:16.731 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-03-09T02:43:16.805 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-03-09T02:43:16.880 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-03-09T02:43:16.949 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-03-09T02:43:17.034 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-03-09T02:43:17.113 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-09T02:43:17.178 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-03-09T02:43:17.284 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-03-09T02:43:17.327 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-09T02:43:17.328 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:17.445 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-03-09T02:43:17.501 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-03-09T02:43:17.596 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-03-09T02:43:17.650 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-03-09T02:43:17.745 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-03-09T02:43:17.747 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:17.800 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-03-09T02:43:17.899 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-03-09T02:43:17.949 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-03-09T02:43:18.033 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-03-09T02:43:18.098 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-03-09T02:43:18.192 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-03-09T02:43:18.235 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-03-09T02:43:18.330 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-09T02:43:18.384 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-03-09T02:43:18.479 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-03-09T02:43:18.543 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-09T02:43:18.628 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-03-09T02:43:18.692 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-03-09T02:43:18.778 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-03-09T02:43:18.831 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-03-09T02:43:18.926 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-03-09T02:43:18.993 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-03-09T02:43:19.087 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-03-09T02:43:19.132 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-03-09T02:43:19.237 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-03-09T02:43:19.280 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-03-09T02:43:19.386 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-03-09T02:43:19.419 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-03-09T02:43:19.526 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-03-09T02:43:19.568 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-03-09T02:43:19.674 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-03-09T02:43:19.717 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-03-09T02:43:19.812 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-03-09T02:43:19.865 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-03-09T02:43:19.970 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-03-09T02:43:20.015 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-03-09T02:43:20.110 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-03-09T02:43:20.143 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-03-09T02:43:20.249 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-03-09T02:43:20.292 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-03-09T02:43:20.398 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-03-09T02:43:20.441 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-03-09T02:43:20.547 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-03-09T02:43:20.590 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-03-09T02:43:20.695 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-03-09T02:43:20.738 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-03-09T02:43:20.843 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-09T02:43:20.876 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-03-09T02:43:20.992 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-03-09T02:43:21.024 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-03-09T02:43:21.150 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-03-09T02:43:21.172 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-03-09T02:43:21.288 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-03-09T02:43:21.310 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-03-09T02:43:21.435 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-03-09T02:43:21.457 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-03-09T02:43:21.582 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-03-09T02:43:21.604 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-03-09T02:43:21.732 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-03-09T02:43:21.754 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-03-09T02:43:21.872 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-03-09T02:43:21.895 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-03-09T02:43:22.023 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-03-09T02:43:22.044 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-03-09T02:43:22.171 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-03-09T02:43:22.183 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-03-09T02:43:22.321 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-03-09T02:43:22.333 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-03-09T02:43:22.463 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-03-09T02:43:22.466 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:22.478 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-03-09T02:43:22.605 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-03-09T02:43:22.627 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-03-09T02:43:22.744 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-03-09T02:43:22.776 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-03-09T02:43:22.921 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-03-09T02:43:22.932 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-03-09T02:43:23.069 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-03-09T02:43:23.082 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-03-09T02:43:23.230 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-03-09T02:43:23.231 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-03-09T02:43:23.378 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-03-09T02:43:23.379 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-03-09T02:43:23.527 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-03-09T02:43:23.528 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-03-09T02:43:23.682 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-03-09T02:43:23.841 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-03-09T02:43:23.842 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-03-09T02:43:23.989 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-03-09T02:43:23.990 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1183} 2021-03-09T02:43:24.139 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-03-09T02:43:24.140 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1184} 2021-03-09T02:43:24.288 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-03-09T02:43:24.290 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-03-09T02:43:24.440 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-03-09T02:43:24.566 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-03-09T02:43:24.588 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-03-09T02:43:24.718 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-03-09T02:43:24.730 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-03-09T02:43:24.867 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-03-09T02:43:24.869 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-03-09T02:43:25.016 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-03-09T02:43:25.018 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-03-09T02:43:25.167 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-03-09T02:43:25.168 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-03-09T02:43:25.305 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-03-09T02:43:25.316 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-03-09T02:43:25.443 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-03-09T02:43:25.465 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-03-09T02:43:25.591 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-03-09T02:43:25.603 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-03-09T02:43:25.741 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-03-09T02:43:25.752 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-03-09T02:43:25.890 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-03-09T02:43:25.892 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-03-09T02:43:26.028 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-03-09T02:43:26.040 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-03-09T02:43:26.177 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-03-09T02:43:26.178 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-03-09T02:43:26.316 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-03-09T02:43:26.327 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-03-09T02:43:26.463 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-03-09T02:43:26.467 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-03-09T02:43:26.468 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:26.607 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-09T02:43:26.618 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-03-09T02:43:26.746 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-03-09T02:43:26.758 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-03-09T02:43:26.897 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-03-09T02:43:26.899 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:26.909 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-03-09T02:43:27.042 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-03-09T02:43:27.054 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-03-09T02:43:27.181 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-03-09T02:43:27.204 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-03-09T02:43:27.330 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-03-09T02:43:27.342 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-03-09T02:43:27.490 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-03-09T02:43:27.491 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-03-09T02:43:27.639 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-03-09T02:43:27.640 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-03-09T02:43:27.777 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-03-09T02:43:27.778 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-03-09T02:43:27.925 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-03-09T02:43:27.927 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-03-09T02:43:28.073 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-03-09T02:43:28.075 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-03-09T02:43:28.212 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-03-09T02:43:28.224 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-03-09T02:43:28.363 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-03-09T02:43:28.364 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-03-09T02:43:28.511 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-03-09T02:43:28.512 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-03-09T02:43:28.661 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-03-09T02:43:28.662 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-03-09T02:43:28.814 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-03-09T02:43:28.816 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-03-09T02:43:28.954 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-03-09T02:43:28.956 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-03-09T02:43:29.093 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-03-09T02:43:29.117 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-03-09T02:43:29.245 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-03-09T02:43:29.267 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-03-09T02:43:29.394 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-03-09T02:43:29.405 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-03-09T02:43:29.532 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-03-09T02:43:29.554 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-03-09T02:43:29.681 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-03-09T02:43:29.692 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-03-09T02:43:29.819 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-03-09T02:43:29.851 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-03-09T02:43:29.966 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-03-09T02:43:29.988 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-03-09T02:43:30.104 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-03-09T02:43:30.147 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-03-09T02:43:30.253 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-03-09T02:43:30.296 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-03-09T02:43:30.393 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-03-09T02:43:30.435 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-03-09T02:43:30.551 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-03-09T02:43:30.688 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-03-09T02:43:30.763 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-03-09T02:43:30.838 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-03-09T02:43:30.985 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-03-09T02:43:31.030 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-03-09T02:43:31.125 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-03-09T02:43:31.168 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-03-09T02:43:31.263 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-03-09T02:43:31.317 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1230, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1230} 2021-03-09T02:43:31.412 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-03-09T02:43:31.456 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/hmunjulu", "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "move_home": false, "name": "hmunjulu", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-03-09T02:43:31.552 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-03-09T02:43:31.606 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-03-09T02:43:31.702 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-03-09T02:43:31.745 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-03-09T02:43:31.851 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-03-09T02:43:31.883 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-03-09T02:43:31.989 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-03-09T02:43:32.091 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-03-09T02:43:32.207 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:32 +0000 (0:00:20.961) 0:00:28.297 ********* 2021-03-09T02:43:32.215 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-03-09T02:43:32.284 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-03-09T02:43:32.307 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:32 +0000 (0:00:00.100) 0:00:28.397 ********* 2021-03-09T02:43:32.325 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-03-09T02:43:32.327 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-03-09T02:43:32.455 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-03-09T02:43:32.456 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-03-09T02:43:32.458 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-03-09T02:43:32.460 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop K 2021-03-09T02:43:32.461 INFO:teuthology.task.ansible.out:D0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9 2021-03-09T02:43:32.463 INFO:teuthology.task.ansible.out:df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRah 2021-03-09T02:43:32.464 INFO:teuthology.task.ansible.out:w fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 2021-03-09T02:43:32.465 INFO:teuthology.task.ansible.out: 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc9795 2021-03-09T02:43:32.467 INFO:teuthology.task.ansible.out:3cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e 2021-03-09T02:43:32.468 INFO:teuthology.task.ansible.out:7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef14 2021-03-09T02:43:32.471 INFO:teuthology.task.ansible.out:80c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d301 2021-03-09T02:43:32.473 INFO:teuthology.task.ansible.out:7b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214 2021-03-09T02:43:32.474 INFO:teuthology.task.ansible.out:bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelope 2021-03-09T02:43:32.476 INFO:teuthology.task.ansible.out:z@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed 2021-03-09T02:43:32.477 INFO:teuthology.task.ansible.out:19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-03-09T02:43:32.505 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:32 +0000 (0:00:00.198) 0:00:28.596 ********* 2021-03-09T02:43:32.528 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:32 +0000 (0:00:00.021) 0:00:28.617 ********* 2021-03-09T02:43:32.599 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-03-09T02:43:32.622 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:32 +0000 (0:00:00.095) 0:00:28.712 ********* 2021-03-09T02:43:32.714 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-03-09T02:43:32.716 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@fedora 2021-03-09T02:43:32.718 INFO:teuthology.task.ansible.out:rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265 2021-03-09T02:43:32.720 INFO:teuthology.task.ansible.out:c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop K 2021-03-09T02:43:32.721 INFO:teuthology.task.ansible.out:D0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9 2021-03-09T02:43:32.723 INFO:teuthology.task.ansible.out:df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRah 2021-03-09T02:43:32.723 INFO:teuthology.task.ansible.out:w fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 2021-03-09T02:43:32.725 INFO:teuthology.task.ansible.out: 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc9795 2021-03-09T02:43:32.727 INFO:teuthology.task.ansible.out:3cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e 2021-03-09T02:43:32.728 INFO:teuthology.task.ansible.out:7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef14 2021-03-09T02:43:32.730 INFO:teuthology.task.ansible.out:80c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d301 2021-03-09T02:43:32.732 INFO:teuthology.task.ansible.out:7b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214 2021-03-09T02:43:32.734 INFO:teuthology.task.ansible.out:bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelope 2021-03-09T02:43:32.736 INFO:teuthology.task.ansible.out:z@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-noteboo 2021-03-09T02:43:32.736 INFO:teuthology.task.ansible.out:k LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed 2021-03-09T02:43:32.738 INFO:teuthology.task.ansible.out:19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}]}, "changed": false} 2021-03-09T02:43:32.766 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:32 +0000 (0:00:00.144) 0:00:28.856 ********* 2021-03-09T02:43:33.282 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-03-09T02:43:33.285 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"after": "c31004c4720a55ac12a95d66c32e64f592877604", "attempts": 1, "before": "c31004c4720a55ac12a95d66c32e64f592877604", "changed": false, "remote_url_changed": false} 2021-03-09T02:43:33.308 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:33 +0000 (0:00:00.541) 0:00:29.398 ********* 2021-03-09T02:43:33.348 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"after": "c31004c4720a55ac12a95d66c32e64f592877604", "attempts": 1, "before": "c31004c4720a55ac12a95d66c32e64f592877604", "changed": false, "remote_url_changed": false} 2021-03-09T02:43:33.374 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:33 +0000 (0:00:00.065) 0:00:29.464 ********* 2021-03-09T02:43:33.537 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-03-09T02:43:33.539 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-03-09T02:43:33.611 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-03-09T02:43:33.675 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-03-09T02:43:33.759 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-03-09T02:43:33.812 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-03-09T02:43:33.885 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-03-09T02:43:33.949 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-03-09T02:43:33.951 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-03-09T02:43:34.023 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-03-09T02:43:34.024 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-03-09T02:43:34.075 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-03-09T02:43:34.150 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-03-09T02:43:34.203 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-03-09T02:43:34.205 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-03-09T02:43:34.287 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-03-09T02:43:34.289 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-03-09T02:43:34.340 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-03-09T02:43:34.413 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-03-09T02:43:34.467 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-03-09T02:43:34.541 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCgaz5AsLB+hPwnHjlf6vktlUQ8iWsKA+hViok+j+uCm0U/iho7O7Qd8E/3tp9yBx10Wkvuz1mmf6v2AO7Qa8+SE1QQPmA+4xLp4xXCqlUGFjnzm/7uQcQZtlWdfBHzQ9jVyPAgUpnGVUtW8PhW0txe8MEtecOO2C3Snx0yuTOx6O11jNvT938QjOeRQWw3X2YFUtMIOiwEhKtqstBIGXlMdaQKfvdlbA9MjyYy6VfK1x8lVcu8umUf/z56YmMkC2iO117/obhdu1XsU9SmjM51wHMOUCXSVqH8E3BWu0yohy4QmZDkdFihyYa+hoS9dOqGC91dFaQc46xrdmV4wqg1 jcaratza@MacBook-Pro\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-03-09T02:43:34.614 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-03-09T02:43:34.625 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:34.689 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-03-09T02:43:34.700 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:34.753 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-03-09T02:43:34.755 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-03-09T02:43:34.828 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-03-09T02:43:34.829 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-03-09T02:43:34.880 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-03-09T02:43:34.882 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-03-09T02:43:34.891 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:34.902 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:34.955 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-03-09T02:43:34.957 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-03-09T02:43:34.966 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:34.977 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:35.030 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-03-09T02:43:35.105 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-03-09T02:43:35.160 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-03-09T02:43:35.162 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-03-09T02:43:35.244 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-03-09T02:43:35.246 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-03-09T02:43:35.298 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-03-09T02:43:35.372 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-03-09T02:43:35.435 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": 2021-03-09T02:43:35.437 INFO:teuthology.task.ansible.out:null, "state": "present", "user": "davidz", "validate_certs": true} 2021-03-09T02:43:35.499 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": 2021-03-09T02:43:35.501 INFO:teuthology.task.ansible.out:null, "state": "present", "user": "davidz", "validate_certs": true} 2021-03-09T02:43:35.574 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-03-09T02:43:35.638 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-03-09T02:43:35.713 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-03-09T02:43:35.756 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-03-09T02:43:35.845 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-03-09T02:43:35.909 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2021-03-09T02:43:35.995 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-03-09T02:43:35.996 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-03-09T02:43:36.059 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-03-09T02:43:36.061 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-03-09T02:43:36.124 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-03-09T02:43:36.126 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-03-09T02:43:36.202 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-03-09T02:43:36.204 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-03-09T02:43:36.255 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-03-09T02:43:36.331 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-03-09T02:43:36.374 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-03-09T02:43:36.458 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-03-09T02:43:36.501 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-03-09T02:43:36.503 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-03-09T02:43:36.596 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46 2021-03-09T02:43:36.597 INFO:teuthology.task.ansible.out:TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-03-09T02:43:36.649 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-03-09T02:43:36.734 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-03-09T02:43:36.777 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-03-09T02:43:36.868 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-03-09T02:43:36.900 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-03-09T02:43:36.997 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-03-09T02:43:37.040 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-03-09T02:43:37.126 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-03-09T02:43:37.179 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-03-09T02:43:37.253 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-03-09T02:43:37.306 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-03-09T02:43:37.308 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-03-09T02:43:37.384 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-03-09T02:43:37.386 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-03-09T02:43:37.428 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-03-09T02:43:37.525 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2021-03-09T02:43:37.558 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-03-09T02:43:37.653 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-03-09T02:43:37.697 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-03-09T02:43:37.794 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-03-09T02:43:37.826 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-03-09T02:43:37.932 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-03-09T02:43:37.964 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-03-09T02:43:38.070 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2021-03-09T02:43:38.081 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-03-09T02:43:38.207 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-09T02:43:38.209 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-03-09T02:43:38.324 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-09T02:43:38.335 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-03-09T02:43:38.461 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-03-09T02:43:38.463 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-03-09T02:43:38.587 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-03-09T02:43:38.589 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-03-09T02:43:38.714 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-03-09T02:43:38.716 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeW 2021-03-09T02:43:38.717 INFO:teuthology.task.ansible.out:D3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-03-09T02:43:38.854 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-03-09T02:43:38.856 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-03-09T02:43:38.857 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-03-09T02:43:38.983 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-03-09T02:43:38.994 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-03-09T02:43:39.122 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-03-09T02:43:39.133 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-03-09T02:43:39.259 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-03-09T02:43:39.270 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-03-09T02:43:39.272 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-09T02:43:39.375 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": nu 2021-03-09T02:43:39.377 INFO:teuthology.task.ansible.out:ll, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-09T02:43:39.397 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-03-09T02:43:39.399 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-03-09T02:43:39.402 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-03-09T02:43:39.513 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-03-09T02:43:39.515 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-03-09T02:43:39.517 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-03-09T02:43:39.524 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-03-09T02:43:39.639 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-03-09T02:43:39.652 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-03-09T02:43:39.779 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-03-09T02:43:39.790 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-03-09T02:43:39.916 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-03-09T02:43:39.918 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-03-09T02:43:40.045 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-03-09T02:43:40.056 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-03-09T02:43:40.174 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-03-09T02:43:40.432 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-03-09T02:43:40.433 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0b 2021-03-09T02:43:40.435 INFO:teuthology.task.ansible.out:sfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-03-09T02:43:40.458 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-03-09T02:43:40.494 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-03-09T02:43:40.607 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-03-09T02:43:40.632 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-03-09T02:43:40.634 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-03-09T02:43:40.736 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-03-09T02:43:40.739 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-03-09T02:43:40.747 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-03-09T02:43:40.873 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-03-09T02:43:40.885 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-03-09T02:43:41.001 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-03-09T02:43:41.022 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-03-09T02:43:41.024 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-03-09T02:43:41.142 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-03-09T02:43:41.144 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-03-09T02:43:41.146 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-03-09T02:43:41.271 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-03-09T02:43:41.283 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-03-09T02:43:41.398 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-03-09T02:43:41.409 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-03-09T02:43:41.535 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-03-09T02:43:41.546 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-09T02:43:41.652 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-03-09T02:43:41.674 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-03-09T02:43:41.789 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-03-09T02:43:41.800 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-03-09T02:43:41.926 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-03-09T02:43:41.938 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-03-09T02:43:42.054 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-03-09T02:43:42.065 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-03-09T02:43:42.182 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-03-09T02:43:42.204 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-03-09T02:43:42.308 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-03-09T02:43:42.330 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-03-09T02:43:42.435 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-03-09T02:43:42.457 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-03-09T02:43:42.561 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-03-09T02:43:42.583 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-03-09T02:43:42.688 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-03-09T02:43:42.711 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-03-09T02:43:42.825 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-03-09T02:43:42.847 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-03-09T02:43:42.952 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-03-09T02:43:42.973 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-03-09T02:43:43.078 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-03-09T02:43:43.121 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-03-09T02:43:43.204 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-03-09T02:43:43.236 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-03-09T02:43:43.331 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-03-09T02:43:43.363 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-03-09T02:43:43.468 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-03-09T02:43:43.489 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-03-09T02:43:43.594 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-03-09T02:43:43.616 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-03-09T02:43:43.732 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-03-09T02:43:43.733 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-03-09T02:43:43.735 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-03-09T02:43:43.859 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-03-09T02:43:43.861 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-03-09T02:43:43.871 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-03-09T02:43:43.998 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-03-09T02:43:43.999 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-03-09T02:43:44.125 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-03-09T02:43:44.137 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-03-09T02:43:44.256 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-03-09T02:43:44.268 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-03-09T02:43:44.386 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2021-03-09T02:43:44.397 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-03-09T02:43:44.398 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-03-09T02:43:44.502 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.locald 2021-03-09T02:43:44.504 INFO:teuthology.task.ansible.out:omain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2021-03-09T02:43:44.524 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-03-09T02:43:44.526 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-03-09T02:43:44.629 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-03-09T02:43:44.631 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-03-09T02:43:44.662 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-03-09T02:43:44.758 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-03-09T02:43:44.790 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-03-09T02:43:44.884 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-03-09T02:43:44.916 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-03-09T02:43:45.000 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-03-09T02:43:45.043 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-03-09T02:43:45.127 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-03-09T02:43:45.169 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-03-09T02:43:45.265 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-03-09T02:43:45.267 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:45.298 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-03-09T02:43:45.396 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-03-09T02:43:45.428 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-03-09T02:43:45.533 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-03-09T02:43:45.566 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-03-09T02:43:45.567 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-03-09T02:43:45.662 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-03-09T02:43:45.663 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-03-09T02:43:45.695 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-03-09T02:43:45.792 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-03-09T02:43:45.824 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-03-09T02:43:45.919 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-03-09T02:43:45.951 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-03-09T02:43:46.057 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-03-09T02:43:46.089 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-03-09T02:43:46.173 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-03-09T02:43:46.215 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-03-09T02:43:46.299 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-03-09T02:43:46.352 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-03-09T02:43:46.427 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-03-09T02:43:46.490 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-03-09T02:43:46.492 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-09T02:43:46.553 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", 2021-03-09T02:43:46.555 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-03-09T02:43:46.616 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-03-09T02:43:46.680 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-03-09T02:43:46.743 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-03-09T02:43:46.796 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-03-09T02:43:46.872 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-03-09T02:43:46.924 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-03-09T02:43:46.998 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-03-09T02:43:47.041 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-03-09T02:43:47.136 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-03-09T02:43:47.168 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-03-09T02:43:47.263 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-03-09T02:43:47.305 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-03-09T02:43:47.389 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-03-09T02:43:47.432 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-03-09T02:43:47.516 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-03-09T02:43:47.559 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-03-09T02:43:47.642 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-03-09T02:43:47.644 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:47.684 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-03-09T02:43:47.779 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-03-09T02:43:47.811 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-03-09T02:43:47.905 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-03-09T02:43:47.939 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-03-09T02:43:48.033 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-03-09T02:43:48.075 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-03-09T02:43:48.160 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-03-09T02:43:48.203 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-03-09T02:43:48.286 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-03-09T02:43:48.329 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-03-09T02:43:48.403 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-03-09T02:43:48.467 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-03-09T02:43:48.531 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-03-09T02:43:48.595 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-03-09T02:43:48.658 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-03-09T02:43:48.733 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-03-09T02:43:48.786 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-03-09T02:43:48.860 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-03-09T02:43:48.913 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-03-09T02:43:48.915 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-03-09T02:43:48.998 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-03-09T02:43:49.000 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-03-09T02:43:49.051 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-03-09T02:43:49.126 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-03-09T02:43:49.191 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-03-09T02:43:49.255 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-03-09T02:43:49.320 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-03-09T02:43:49.395 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-03-09T02:43:49.458 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-03-09T02:43:49.543 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-03-09T02:43:49.599 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-03-09T02:43:49.683 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-03-09T02:43:49.749 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-03-09T02:43:49.824 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-03-09T02:43:49.880 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-03-09T02:43:49.976 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-03-09T02:43:50.039 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-03-09T02:43:50.127 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-03-09T02:43:50.171 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-03-09T02:43:50.269 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-03-09T02:43:50.325 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-03-09T02:43:50.414 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-03-09T02:43:50.502 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-03-09T02:43:50.576 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-03-09T02:43:50.579 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:50.630 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-03-09T02:43:50.726 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQKiUAlB8dxudAY60M7jnIJvs9yb2pNaT+BVv1Vk1kMs0XjPjcw9cyT3DtQ1xGLF367phRi/cfQbXOYbHG+uyBzQ4EEnBFvziktWRx4mfeWYxKjkjbfkVfpKPsxztqCbwm6+NYzMxGK2cExN7jUHnghHAjfA0IUpVVg9peBD3kUveLwS+UxuJAnuj+G4480LS6GDyLrhFEXU7uOIIlW4S9oLnU6vJL6GAzwyLE4ijTA8Ek03SJ01Ey/gpU9AqnQNp9u12l/7sJely6sI9pAuIzmS8Ht324hexxSWN5sFjXgLvLfFK83ziDBdrFBr0WXnWsFdRpHs2CwSq1JTkByqbN hmunjulu@hmunjulu.remote.csb", "key_options": null, "keyfile": "/home/hmunjulu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hmunjulu", "validate_certs": true} 2021-03-09T02:43:50.779 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-03-09T02:43:50.865 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-03-09T02:43:50.908 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-03-09T02:43:50.982 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev", "key_options": null, "keyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-03-09T02:43:51.025 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-03-09T02:43:51.109 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-03-09T02:43:51.151 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-03-09T02:43:51.236 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-03-09T02:43:51.280 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_options" 2021-03-09T02:43:51.282 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-03-09T02:43:51.374 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_options" 2021-03-09T02:43:51.376 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-03-09T02:43:51.406 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-03-09T02:43:51.505 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:51 +0000 (0:00:18.131) 0:00:47.595 ********* 2021-03-09T02:43:51.516 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDr6H7hmtGCUrO3GppgMHwbmA4XT/Vipz/uLG/N8h6ZxtaLQKpCefRpdKcFlOdDwxkTmT5q9vmXQ8nl95lTGq+eeMxl4ho104ZD9e9AIYmgYqFy7/qNP8gkqduIPlypLL9m5rElTcGd7zqfMqgljMvAmnXLtZNJlIuFtszVU5WPI1QCwfQ3t5aHF6j66bpVoN9uWItVGB1LApQz9TtViWsQ9Hl8X4YauJToO+TkuSX+H/OGH7ZC6CPOzNZb3b3vF/jLMnTzXl86PJs1se4DjKkRGLVCVbr2WhLwQeZ8TEn3Hg1Gn5X8zNM5cTo6U8SnXBAe9ZYbuyfvghrNLdlZpmN+8l0/l5zC3z+N+sEzklXwHw4MQLF+KSloVFd9NN/kyenufTZAyp8VxllZ9TgdC+CPORIWyzmu1elOtYo7LNuE6CIh/Zm9B54SO6hVHhVoOCqe/dIyjeHJA0WKIZblK1XZEvWuu0E2T3jhxfe1GICyTukeB8WoBQH9DZ1yzMPt59E= nia@redhat.com", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-03-09T02:43:51.602 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:51 +0000 (0:00:00.096) 0:00:47.692 ********* 2021-03-09T02:43:51.622 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.633 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.634 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.646 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.658 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.662 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.664 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.675 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.676 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.687 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.698 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.700 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.710 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.721 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.724 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.733 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.734 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.745 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.809 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-03-09T02:43:51.820 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.832 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.874 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-03-09T02:43:51.877 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.887 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:51.961 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-03-09T02:43:52.015 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-03-09T02:43:52.089 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfad 2021-03-09T02:43:52.091 INFO:teuthology.task.ansible.out:a28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.100 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.112 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.113 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.123 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.125 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.134 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.145 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.147 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf 2021-03-09T02:43:52.149 INFO:teuthology.task.ansible.out:370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.158 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.160 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f8 2021-03-09T02:43:52.161 INFO:teuthology.task.ansible.out:8afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.170 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.172 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1 2021-03-09T02:43:52.174 INFO:teuthology.task.ansible.out:a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\ndavid@ubuntu-2004 sz2UXQopIP7TJ3t/XtOylw 515687a2ace008dbc318525cb273afed0887aa03ae9ca9b0933ca4792ef4addc\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.183 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.184 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.194 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.196 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.206 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.207 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.208 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.209 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.221 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.223 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.234 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.235 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.246 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.249 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.258 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.260 INFO:teuthology.task.ansible.out: skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.271 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.272 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.283 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.285 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.296 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.298 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.309 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.311 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.320 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.322 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.332 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.335 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.338 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.347 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.349 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.358 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.360 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.370 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.371 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.382 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.384 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.386 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.397 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.399 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.400 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.411 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.413 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.424 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.426 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.437 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.438 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.449 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.451 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.451 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.462 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.464 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.464 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.465 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.476 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.478 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.489 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.491 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.502 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.504 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.515 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.516 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.528 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.531 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.539 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.542 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.543 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.553 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.555 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.566 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.570 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.579 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.581 INFO:teuthology.task.ansible.out: skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.592 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.593 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.604 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.606 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.617 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.618 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.629 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.630 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.642 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.644 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.653 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.655 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.666 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.668 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.681 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.682 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.693 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.695 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost QwKbKHfNAOenVV0mkO8NXw 40d74abb20ebba9eaca5e58f6c821670d23f5db5405fc6e3072b985a397c0104"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.696 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.707 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.708 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.718 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.720 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.732 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.733 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.734 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.745 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.746 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.756 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.757 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.758 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.759 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.769 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.770 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.772 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.783 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.785 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.795 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.797 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.798 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.809 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.810 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.812 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.814 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.825 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.827 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.828 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.840 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.842 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.853 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.855 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.855 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.867 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.868 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.880 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.891 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.893 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.903 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.905 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.916 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.918 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.928 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.930 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.941 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.942 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.953 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.953 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.954 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.955 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.965 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.967 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.978 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.980 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:52.993 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad huZBxlH4VpGff0Q68m/2Rg 43cc9c90a55fa3395862e679b60c5e6c016a59f186e7e8798579cd6046cc4ec8"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:53.012 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:53.014 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@thinkpad 5H//Tz6a0uBQDJiYlp9bIA 5ad4213c5f177c9d3586e58770208dd3e18f57d78f12d5d1a9dfa541d5c5c957\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat +vB2tfopNqiNnGugVmX3ew f9f6878bb5e6bb2353fa8da7d76e2d621e9fbf89a4c7583d7265cb984e231a99"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:53.036 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:53 +0000 (0:00:01.434) 0:00:49.126 ********* 2021-03-09T02:43:53.094 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:53 +0000 (0:00:00.057) 0:00:49.184 ********* 2021-03-09T02:43:53.113 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:53.137 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:53 +0000 (0:00:00.043) 0:00:49.227 ********* 2021-03-09T02:43:53.164 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:43:53.192 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:43:53 +0000 (0:00:00.055) 0:00:49.282 ********* 2021-03-09T02:43:53.348 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [smithi092.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-03-09T02:43:53.411 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-03-09T02:43:53.476 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-03-09T02:43:53.551 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-03-09T02:43:53.553 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:53.615 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-03-09T02:43:53.689 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-03-09T02:43:53.764 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-03-09T02:43:53.828 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-03-09T02:43:53.901 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-03-09T02:43:53.965 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-03-09T02:43:54.038 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-03-09T02:43:54.101 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-03-09T02:43:54.176 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-03-09T02:43:54.240 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-03-09T02:43:54.313 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-03-09T02:43:54.376 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-03-09T02:43:54.450 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-03-09T02:43:54.515 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-03-09T02:43:54.590 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-03-09T02:43:54.654 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-03-09T02:43:54.729 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-03-09T02:43:54.792 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-03-09T02:43:54.859 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-03-09T02:43:54.933 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-03-09T02:43:54.996 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-03-09T02:43:55.070 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-03-09T02:43:55.134 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-03-09T02:43:55.218 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-03-09T02:43:55.274 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-03-09T02:43:55.358 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-03-09T02:43:55.410 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-03-09T02:43:55.494 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-03-09T02:43:55.548 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-03-09T02:43:55.642 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-03-09T02:43:55.685 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-03-09T02:43:55.789 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-03-09T02:43:55.831 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-03-09T02:43:55.926 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-03-09T02:43:55.969 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-03-09T02:43:56.064 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-03-09T02:43:56.107 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-03-09T02:43:56.201 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-03-09T02:43:56.243 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-03-09T02:43:56.338 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2021-03-09T02:43:56.381 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-03-09T02:43:56.475 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-03-09T02:43:56.518 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-03-09T02:43:56.613 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-03-09T02:43:56.655 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-03-09T02:43:56.750 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-03-09T02:43:56.792 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-03-09T02:43:56.887 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-03-09T02:43:56.930 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-03-09T02:43:57.035 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-03-09T02:43:57.067 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-03-09T02:43:57.172 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-03-09T02:43:57.204 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-03-09T02:43:57.310 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-03-09T02:43:57.353 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-03-09T02:43:57.437 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-03-09T02:43:57.480 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-03-09T02:43:57.575 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-03-09T02:43:57.618 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-03-09T02:43:57.714 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-03-09T02:43:57.716 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:57.759 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-03-09T02:43:57.854 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-03-09T02:43:57.897 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-03-09T02:43:57.992 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-03-09T02:43:58.034 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-03-09T02:43:58.129 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-03-09T02:43:58.163 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-03-09T02:43:58.269 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-03-09T02:43:58.312 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-03-09T02:43:58.407 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-03-09T02:43:58.409 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:58.450 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-03-09T02:43:58.452 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:58.546 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-03-09T02:43:58.590 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-03-09T02:43:58.685 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-03-09T02:43:58.727 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-03-09T02:43:58.823 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-03-09T02:43:58.866 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-03-09T02:43:58.962 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-03-09T02:43:59.005 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-03-09T02:43:59.100 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-03-09T02:43:59.143 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-03-09T02:43:59.145 INFO:teuthology.task.ansible.out: 2021-03-09T02:43:59.239 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-03-09T02:43:59.281 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-03-09T02:43:59.376 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-03-09T02:43:59.419 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-03-09T02:43:59.515 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-03-09T02:43:59.557 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-03-09T02:43:59.652 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-03-09T02:43:59.694 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-03-09T02:43:59.790 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-03-09T02:43:59.833 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-03-09T02:43:59.929 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-03-09T02:43:59.981 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-03-09T02:44:00.026 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:00 +0000 (0:00:06.833) 0:00:56.116 ********* 2021-03-09T02:44:00.066 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-03-09T02:44:00.129 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:00 +0000 (0:00:00.103) 0:00:56.220 ********* 2021-03-09T02:44:00.432 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-03-09T02:44:00.434 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "checksum": "1d9711ff879c231c67cb70d00ca0088797a377d0", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "80a55def4b505d08aff3687f81f74dc2", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615257840.1109767-8794-97128557506643/source", "state": "file", "uid": 0} 2021-03-09T02:44:00.457 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:00 +0000 (0:00:00.327) 0:00:56.547 ********* 2021-03-09T02:44:00.537 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "checksum": "1d9711ff879c231c67cb70d00ca0088797a377d0", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "80a55def4b505d08aff3687f81f74dc2", "mode": "0644", "owner": "root", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615257840.2226722-8811-80055297972434/source", "state": "file", "uid": 0} 2021-03-09T02:44:00.561 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:00 +0000 (0:00:00.103) 0:00:56.651 ********* 2021-03-09T02:44:00.567 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-03-09T02:44:00.569 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => { "msg": "Host smithi092.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-03-09T02:44:00.595 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:00 +0000 (0:00:00.034) 0:00:56.685 ********* 2021-03-09T02:44:00.634 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => { "msg": "Host smithi144.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2021-03-09T02:44:00.660 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:00 +0000 (0:00:00.064) 0:00:56.750 ********* 2021-03-09T02:44:00.680 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi092.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-03-09T02:44:00.722 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:00 +0000 (0:00:00.059) 0:00:56.810 ********* 2021-03-09T02:44:00.753 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2021-03-09T02:44:00.786 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:00 +0000 (0:00:00.061) 0:00:56.871 ********* 2021-03-09T02:44:00.788 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-03-09T02:44:00.789 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:00.820 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:00 +0000 (0:00:00.039) 0:00:56.910 ********* 2021-03-09T02:44:00.849 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:00.874 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:00 +0000 (0:00:00.052) 0:00:56.963 ********* 2021-03-09T02:44:00.891 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2021-03-09T02:44:00.894 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:00.918 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:00 +0000 (0:00:00.044) 0:00:57.008 ********* 2021-03-09T02:44:00.987 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.013 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.094) 0:00:57.103 ********* 2021-03-09T02:44:01.020 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2021-03-09T02:44:01.021 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.054 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.040) 0:00:57.144 ********* 2021-03-09T02:44:01.085 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.110 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.056) 0:00:57.200 ********* 2021-03-09T02:44:01.127 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-03-09T02:44:01.128 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.154 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.044) 0:00:57.244 ********* 2021-03-09T02:44:01.193 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.220 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.065) 0:00:57.310 ********* 2021-03-09T02:44:01.232 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-03-09T02:44:01.233 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.260 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.040) 0:00:57.351 ********* 2021-03-09T02:44:01.300 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.327 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.066) 0:00:57.417 ********* 2021-03-09T02:44:01.344 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-03-09T02:44:01.346 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.369 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.041) 0:00:57.459 ********* 2021-03-09T02:44:01.396 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.422 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.052) 0:00:57.512 ********* 2021-03-09T02:44:01.452 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.476 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.054) 0:00:57.566 ********* 2021-03-09T02:44:01.494 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.519 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.043) 0:00:57.609 ********* 2021-03-09T02:44:01.557 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-03-09T02:44:01.559 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.581 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.061) 0:00:57.671 ********* 2021-03-09T02:44:01.599 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.623 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.041) 0:00:57.713 ********* 2021-03-09T02:44:01.625 INFO:teuthology.task.ansible.out: 2021-03-09T02:44:01.655 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-03-09T02:44:01.657 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:01.682 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.058) 0:00:57.772 ********* 2021-03-09T02:44:01.710 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:01.712 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:01.738 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.055) 0:00:57.828 ********* 2021-03-09T02:44:01.756 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-03-09T02:44:01.758 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.781 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.043) 0:00:57.871 ********* 2021-03-09T02:44:01.820 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.858 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.077) 0:00:57.948 ********* 2021-03-09T02:44:01.868 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-03-09T02:44:01.870 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.906 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.047) 0:00:57.996 ********* 2021-03-09T02:44:01.944 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:01.970 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:01 +0000 (0:00:00.064) 0:00:58.060 ********* 2021-03-09T02:44:01.987 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-03-09T02:44:01.989 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:02.012 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.042) 0:00:58.102 ********* 2021-03-09T02:44:02.031 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:02.061 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.047) 0:00:58.151 ********* 2021-03-09T02:44:02.079 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-03-09T02:44:02.081 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.105 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.044) 0:00:58.195 ********* 2021-03-09T02:44:02.144 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.170 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.064) 0:00:58.260 ********* 2021-03-09T02:44:02.186 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-03-09T02:44:02.188 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:02.214 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.044) 0:00:58.305 ********* 2021-03-09T02:44:02.243 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:02.268 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.053) 0:00:58.358 ********* 2021-03-09T02:44:02.286 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-03-09T02:44:02.287 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.311 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.042) 0:00:58.401 ********* 2021-03-09T02:44:02.340 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.365 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.053) 0:00:58.455 ********* 2021-03-09T02:44:02.393 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-03-09T02:44:02.395 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.418 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.052) 0:00:58.508 ********* 2021-03-09T02:44:02.436 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.463 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.045) 0:00:58.553 ********* 2021-03-09T02:44:02.493 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-03-09T02:44:02.495 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.524 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.060) 0:00:58.614 ********* 2021-03-09T02:44:02.543 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.545 INFO:teuthology.task.ansible.out: 2021-03-09T02:44:02.571 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.046) 0:00:58.661 ********* 2021-03-09T02:44:02.590 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-03-09T02:44:02.592 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.616 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.045) 0:00:58.706 ********* 2021-03-09T02:44:02.644 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.673 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.056) 0:00:58.763 ********* 2021-03-09T02:44:02.691 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-03-09T02:44:02.693 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:02.719 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.045) 0:00:58.809 ********* 2021-03-09T02:44:02.738 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:02.767 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.048) 0:00:58.857 ********* 2021-03-09T02:44:02.795 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-03-09T02:44:02.796 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.822 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.054) 0:00:58.912 ********* 2021-03-09T02:44:02.840 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.866 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.043) 0:00:58.956 ********* 2021-03-09T02:44:02.893 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-03-09T02:44:02.895 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.920 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.054) 0:00:59.010 ********* 2021-03-09T02:44:02.939 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:02.965 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:02 +0000 (0:00:00.044) 0:00:59.055 ********* 2021-03-09T02:44:02.992 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-03-09T02:44:02.994 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.020 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.055) 0:00:59.110 ********* 2021-03-09T02:44:03.038 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.066 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.046) 0:00:59.156 ********* 2021-03-09T02:44:03.097 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-03-09T02:44:03.099 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.126 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.060) 0:00:59.216 ********* 2021-03-09T02:44:03.148 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.178 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.051) 0:00:59.268 ********* 2021-03-09T02:44:03.195 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-03-09T02:44:03.196 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.227 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.049) 0:00:59.317 ********* 2021-03-09T02:44:03.255 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.280 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.053) 0:00:59.370 ********* 2021-03-09T02:44:03.297 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-03-09T02:44:03.298 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.323 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.043) 0:00:59.413 ********* 2021-03-09T02:44:03.353 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.381 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.057) 0:00:59.471 ********* 2021-03-09T02:44:03.397 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-03-09T02:44:03.400 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:03.431 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.049) 0:00:59.521 ********* 2021-03-09T02:44:03.450 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:03.477 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.046) 0:00:59.567 ********* 2021-03-09T02:44:03.516 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-03-09T02:44:03.517 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.542 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.065) 0:00:59.632 ********* 2021-03-09T02:44:03.560 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.585 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.042) 0:00:59.675 ********* 2021-03-09T02:44:03.614 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-03-09T02:44:03.616 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.646 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.060) 0:00:59.736 ********* 2021-03-09T02:44:03.665 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.691 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.044) 0:00:59.781 ********* 2021-03-09T02:44:03.739 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2021-03-09T02:44:03.741 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.767 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.076) 0:00:59.857 ********* 2021-03-09T02:44:03.785 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.813 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.045) 0:00:59.903 ********* 2021-03-09T02:44:03.862 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-03-09T02:44:03.864 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.891 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.078) 0:00:59.981 ********* 2021-03-09T02:44:03.900 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.911 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.938 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.046) 0:01:00.028 ********* 2021-03-09T02:44:03.965 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-03-09T02:44:03.967 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:03.993 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:03 +0000 (0:00:00.054) 0:01:00.083 ********* 2021-03-09T02:44:04.021 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.023 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.047 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.054) 0:01:00.137 ********* 2021-03-09T02:44:04.068 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-03-09T02:44:04.070 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.102 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.055) 0:01:00.192 ********* 2021-03-09T02:44:04.132 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.157 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.055) 0:01:00.247 ********* 2021-03-09T02:44:04.299 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-03-09T02:44:04.300 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.018866", "end": "2021-03-09 02:44:03.827896", "rc": 0, "start": "2021-03-09 02:44:03.809030", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-03-09T02:44:04.330 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.172) 0:01:00.420 ********* 2021-03-09T02:44:04.360 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.015309", "end": "2021-03-09 02:44:04.084969", "rc": 0, "start": "2021-03-09 02:44:04.069660", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-03-09T02:44:04.388 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.058) 0:01:00.478 ********* 2021-03-09T02:44:04.405 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-03-09T02:44:04.407 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.436 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.048) 0:01:00.526 ********* 2021-03-09T02:44:04.476 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.507 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.070) 0:01:00.597 ********* 2021-03-09T02:44:04.538 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-03-09T02:44:04.540 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.571 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.063) 0:01:00.661 ********* 2021-03-09T02:44:04.591 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.628 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.056) 0:01:00.717 ********* 2021-03-09T02:44:04.677 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-03-09T02:44:04.678 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.690 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.717 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.089) 0:01:00.807 ********* 2021-03-09T02:44:04.725 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.737 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.739 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.765 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.047) 0:01:00.855 ********* 2021-03-09T02:44:04.791 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-03-09T02:44:04.793 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.819 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.054) 0:01:00.909 ********* 2021-03-09T02:44:04.847 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.871 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.052) 0:01:00.961 ********* 2021-03-09T02:44:04.888 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2021-03-09T02:44:04.890 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.913 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.041) 0:01:01.003 ********* 2021-03-09T02:44:04.953 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:04.986 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:04 +0000 (0:00:00.072) 0:01:01.076 ********* 2021-03-09T02:44:05.003 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-03-09T02:44:05.007 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:05.031 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:05 +0000 (0:00:00.045) 0:01:01.121 ********* 2021-03-09T02:44:05.070 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:05.098 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:05 +0000 (0:00:00.066) 0:01:01.188 ********* 2021-03-09T02:44:05.105 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:05.130 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:05 +0000 (0:00:00.032) 0:01:01.220 ********* 2021-03-09T02:44:05.179 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:05.203 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:05 +0000 (0:00:00.073) 0:01:01.293 ********* 2021-03-09T02:44:05.211 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-03-09T02:44:05.212 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:05.242 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:05 +0000 (0:00:00.039) 0:01:01.332 ********* 2021-03-09T02:44:05.282 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:05.313 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:05 +0000 (0:00:00.070) 0:01:01.403 ********* 2021-03-09T02:44:05.767 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-03-09T02:44:05.768 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.312262", "end": "2021-03-09 02:44:05.500331", "rc": 0, "start": "2021-03-09 02:44:05.188069", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:44:05.794 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:05 +0000 (0:00:00.480) 0:01:01.884 ********* 2021-03-09T02:44:05.862 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-03-09T02:44:05.864 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:05.892 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:05 +0000 (0:00:00.098) 0:01:01.982 ********* 2021-03-09T02:44:05.984 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-03-09T02:44:05.985 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.592325", "end": "2021-03-09 02:44:05.500412", "rc": 0, "start": "2021-03-09 02:44:04.908087", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:44:06.009 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:06 +0000 (0:00:00.117) 0:01:02.099 ********* 2021-03-09T02:44:06.078 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-03-09T02:44:06.080 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:06.102 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:06 +0000 (0:00:00.093) 0:01:02.192 ********* 2021-03-09T02:44:06.109 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-03-09T02:44:06.111 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.024275", "end": "2021-03-09 02:44:05.824738", "rc": 0, "start": "2021-03-09 02:44:05.800463", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:44:06.135 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:06 +0000 (0:00:00.032) 0:01:02.225 ********* 2021-03-09T02:44:06.404 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.016722", "end": "2021-03-09 02:44:05.932701", "rc": 0, "start": "2021-03-09 02:44:05.915979", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:44:06.438 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:06 +0000 (0:00:00.303) 0:01:02.528 ********* 2021-03-09T02:44:09.377 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-03-09T02:44:09.378 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:44:09.400 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:09 +0000 (0:00:02.961) 0:01:05.490 ********* 2021-03-09T02:44:09.427 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:44:09.452 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:09 +0000 (0:00:00.051) 0:01:05.542 ********* 2021-03-09T02:44:10.445 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-03-09T02:44:10.447 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:44:10.470 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:10 +0000 (0:00:01.018) 0:01:06.560 ********* 2021-03-09T02:44:10.487 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:44:10.513 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:10 +0000 (0:00:00.043) 0:01:06.603 ********* 2021-03-09T02:44:10.543 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-03-09T02:44:10.545 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:10.568 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:10 +0000 (0:00:00.054) 0:01:06.658 ********* 2021-03-09T02:44:10.585 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:10.615 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:10 +0000 (0:00:00.047) 0:01:06.705 ********* 2021-03-09T02:44:11.187 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-03-09T02:44:11.196 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2021-03-09T02:44:11.212 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:11 +0000 (0:00:00.596) 0:01:07.302 ********* 2021-03-09T02:44:11.232 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:11 +0000 (0:00:00.020) 0:01:07.322 ********* 2021-03-09T02:44:12.098 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-03-09T02:44:12.100 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:44:12.121 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:12 +0000 (0:00:00.888) 0:01:08.211 ********* 2021-03-09T02:44:12.263 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:44:12.290 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:12 +0000 (0:00:00.168) 0:01:08.380 ********* 2021-03-09T02:44:12.619 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-03-09T02:44:12.621 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-03-09T02:44:12.915 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-03-09T02:44:12.916 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2021-03-09T02:44:13.277 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-03-09T02:44:13.301 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:13 +0000 (0:00:01.011) 0:01:09.391 ********* 2021-03-09T02:44:13.309 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2021-03-09T02:44:13.425 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-03-09T02:44:13.427 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-03-09T02:44:13.449 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:13 +0000 (0:00:00.147) 0:01:09.539 ********* 2021-03-09T02:44:13.517 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-03-09T02:44:13.519 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2021-03-09T02:44:13.544 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:13 +0000 (0:00:00.095) 0:01:09.634 ********* 2021-03-09T02:44:13.675 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2021-03-09T02:44:13.698 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:13 +0000 (0:00:00.153) 0:01:09.788 ********* 2021-03-09T02:44:13.799 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-03-09T02:44:13.801 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-03-09T02:44:14.062 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 13355, "state": "file", "uid": 0} 2021-03-09T02:44:14.083 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-03-09T02:44:14.347 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2021-03-09T02:44:14.358 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-03-09T02:44:14.384 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:14 +0000 (0:00:00.685) 0:01:10.474 ********* 2021-03-09T02:44:14.466 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-03-09T02:44:14.468 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-03-09T02:44:14.494 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:14 +0000 (0:00:00.110) 0:01:10.584 ********* 2021-03-09T02:44:14.575 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-03-09T02:44:14.576 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:14.609 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:14 +0000 (0:00:00.114) 0:01:10.699 ********* 2021-03-09T02:44:14.626 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-03-09T02:44:14.628 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2021-03-09T02:44:14.653 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:14 +0000 (0:00:00.044) 0:01:10.743 ********* 2021-03-09T02:44:14.680 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-03-09T02:44:14.681 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:14.704 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:14 +0000 (0:00:00.051) 0:01:10.794 ********* 2021-03-09T02:44:14.742 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2021-03-09T02:44:14.744 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2021-03-09T02:44:14.769 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:14 +0000 (0:00:00.065) 0:01:10.859 ********* 2021-03-09T02:44:14.786 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-03-09T02:44:14.788 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:14.798 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:14.800 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:14.824 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:14 +0000 (0:00:00.054) 0:01:10.914 ********* 2021-03-09T02:44:14.853 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2021-03-09T02:44:14.855 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:14.880 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:14 +0000 (0:00:00.055) 0:01:10.970 ********* 2021-03-09T02:44:14.899 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-03-09T02:44:14.901 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:14.925 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:14 +0000 (0:00:00.044) 0:01:11.015 ********* 2021-03-09T02:44:14.952 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-03-09T02:44:14.954 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:14.977 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:14 +0000 (0:00:00.052) 0:01:11.067 ********* 2021-03-09T02:44:15.066 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-03-09T02:44:15.068 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:15.077 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:15.079 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:15.089 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:15.113 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:15 +0000 (0:00:00.135) 0:01:11.203 ********* 2021-03-09T02:44:15.181 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:15.205 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:15 +0000 (0:00:00.091) 0:01:11.295 ********* 2021-03-09T02:44:15.876 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2021-03-09T02:44:15.878 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:44:15.900 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:15 +0000 (0:00:00.695) 0:01:11.990 ********* 2021-03-09T02:44:16.187 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:44:16.215 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:16 +0000 (0:00:00.314) 0:01:12.305 ********* 2021-03-09T02:44:16.264 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-03-09T02:44:16.266 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-03-09T02:44:16.292 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:16 +0000 (0:00:00.077) 0:01:12.382 ********* 2021-03-09T02:44:16.585 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2021-03-09T02:44:16.609 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:16 +0000 (0:00:00.316) 0:01:12.699 ********* 2021-03-09T02:44:16.657 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-03-09T02:44:16.659 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-03-09T02:44:16.683 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:16 +0000 (0:00:00.073) 0:01:12.773 ********* 2021-03-09T02:44:16.878 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-03-09T02:44:16.880 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-03-09T02:44:16.983 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-03-09T02:44:17.007 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:17 +0000 (0:00:00.324) 0:01:13.097 ********* 2021-03-09T02:44:17.013 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-03-09T02:44:17.015 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2021-03-09T02:44:17.040 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:17 +0000 (0:00:00.032) 0:01:13.130 ********* 2021-03-09T02:44:17.077 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-03-09T02:44:17.079 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:17.101 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:17 +0000 (0:00:00.060) 0:01:13.191 ********* 2021-03-09T02:44:17.168 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-03-09T02:44:17.170 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:17.192 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:17 +0000 (0:00:00.091) 0:01:13.282 ********* 2021-03-09T02:44:17.221 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-03-09T02:44:17.223 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2021-03-09T02:44:17.276 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:17.299 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:17 +0000 (0:00:00.107) 0:01:13.389 ********* 2021-03-09T02:44:17.336 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-03-09T02:44:17.338 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2021-03-09T02:44:17.362 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:17 +0000 (0:00:00.062) 0:01:13.452 ********* 2021-03-09T02:44:17.431 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-03-09T02:44:17.432 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:17.454 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:17 +0000 (0:00:00.092) 0:01:13.545 ********* 2021-03-09T02:44:17.532 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2021-03-09T02:44:17.534 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-03-09T02:44:17.560 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:17 +0000 (0:00:00.105) 0:01:13.650 ********* 2021-03-09T02:44:17.627 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-03-09T02:44:17.629 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:17.650 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:17 +0000 (0:00:00.090) 0:01:13.741 ********* 2021-03-09T02:44:17.677 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-03-09T02:44:17.679 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-03-09T02:44:17.703 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:17 +0000 (0:00:00.051) 0:01:13.793 ********* 2021-03-09T02:44:18.033 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2021-03-09T02:44:18.063 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.360) 0:01:14.153 ********* 2021-03-09T02:44:18.333 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-03-09T02:44:18.334 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Tue 2021-03-09 02:42:08 UTC", "ActiveEnterTimestampMonotonic": "16670288", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "systemd-tmpfiles-setup.service var-run.mount local-fs.target -.mount sysinit.target nss-lookup.target time-sync.target network.target systemd-journald.socket remote-fs.target basic.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2021-03-09 02:42:08 UTC", "AssertTimestampMonotonic": "16669419", "Before": "shutdown.target getty@tty1.service multi-user.target xdm.service plymouth-quit.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2021-03-09 02:42:08 UTC", "ConditionTimestampMonotonic": "16669419", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-ser 2021-03-09T02:44:18.336 INFO:teuthology.task.ansible.out:ver.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1310", "ExecMainStartTimestamp": "Tue 2021-03-09 02:42:08 UTC", "ExecMainStartTimestampMonotonic": "16670251", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2021-03-09 02:42:08 UTC", "InactiveExitTimestampMonotonic": "16670288", "InvocationID": "bac9aceb6d304bbcbc3071a1a124a9f1", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "i 2021-03-09T02:44:18.337 INFO:teuthology.task.ansible.out:nfinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1310", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no" 2021-03-09T02:44:18.339 INFO:teuthology.task.ansible.out:, "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2021-03-09 02:42:08 UTC", "StateChangeTimestampMonotonic": "16670288", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2021-03-09 02:42:08 UTC", "WatchdogTimestampMonotonic": "16670287", "WatchdogUSec": "0"}} ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nagios-nrpe-server", "state": "started", "status": {"ActiveEnterTimestamp": "Tue 2021-03-09 02:42:14 UTC", "ActiveEnterTimestampMonotonic": "17477093", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target nss-lookup.target var-run.mount sysinit.target basic.target local-fs.target systemd-journald.socket -.mount network.target systemd-tmpfiles-setup.service 2021-03-09T02:44:18.341 INFO:teuthology.task.ansible.out: system.slice time-sync.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2021-03-09 02:42:14 UTC", "AssertTimestampMonotonic": "17476366", "Before": "getty@tty1.service multi-user.target plymouth-quit.service shutdown.target xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2021-03-09 02:42:14 UTC", "ConditionTimestampMonotonic": "17476365", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nagios-nrpe-server.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Plugin Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "DynamicUser": "no", "EnvironmentFile": "/etc/default/nagios-nrpe-server (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1177", "ExecMainStartTimestamp": "Tue 2021-03-09 02:42:14 UTC", "ExecMainStartTimestampMonotonic": 2021-03-09T02:44:18.343 INFO:teuthology.task.ansible.out:"17477067", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_OPTS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nagios/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/nagios-nrpe-server.service", "GID": "118", "Group": "nagios", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "nagios-nrpe-server.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2021-03-09 02:42:14 UTC", "InactiveExitTimestampMonotonic": "17477093", "InvocationID": "3196ebb7444149d9925216bf3c86b192", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "102 2021-03-09T02:44:18.344 INFO:teuthology.task.ansible.out:4", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1177", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "nagios-nrpe-server.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "OnFailureJobMode": "replace", "PIDFile": "/var/run/nagios/nrpe.pid", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "-.mount sysinit.target system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "Star 2021-03-09T02:44:18.347 INFO:teuthology.task.ansible.out:tLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2021-03-09 02:42:14 UTC", "StateChangeTimestampMonotonic": "17477093", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UID": "109", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "User": "nagios", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2021-03-09 02:42:14 UTC", "WatchdogTimestampMonotonic": "17477092", "WatchdogUSec": "0"}} 2021-03-09T02:44:18.363 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.299) 0:01:14.453 ********* 2021-03-09T02:44:18.384 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.021) 0:01:14.474 ********* 2021-03-09T02:44:18.433 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-03-09T02:44:18.434 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.460 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.075) 0:01:14.550 ********* 2021-03-09T02:44:18.467 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.494 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.034) 0:01:14.585 ********* 2021-03-09T02:44:18.532 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-03-09T02:44:18.534 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.557 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.062) 0:01:14.647 ********* 2021-03-09T02:44:18.574 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.602 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.044) 0:01:14.692 ********* 2021-03-09T02:44:18.629 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-03-09T02:44:18.631 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.656 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.053) 0:01:14.745 ********* 2021-03-09T02:44:18.673 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.699 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.043) 0:01:14.789 ********* 2021-03-09T02:44:18.726 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-03-09T02:44:18.727 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.752 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.052) 0:01:14.842 ********* 2021-03-09T02:44:18.771 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.796 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.044) 0:01:14.886 ********* 2021-03-09T02:44:18.824 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-03-09T02:44:18.825 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.849 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.052) 0:01:14.939 ********* 2021-03-09T02:44:18.867 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.893 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.044) 0:01:14.983 ********* 2021-03-09T02:44:18.921 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-03-09T02:44:18.923 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.950 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.056) 0:01:15.040 ********* 2021-03-09T02:44:18.969 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:18.994 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:18 +0000 (0:00:00.043) 0:01:15.084 ********* 2021-03-09T02:44:19.021 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-03-09T02:44:19.022 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:19.047 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.053) 0:01:15.137 ********* 2021-03-09T02:44:19.065 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:19.090 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.042) 0:01:15.180 ********* 2021-03-09T02:44:19.117 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-03-09T02:44:19.119 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:19.143 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.053) 0:01:15.233 ********* 2021-03-09T02:44:19.160 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:19.188 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.045) 0:01:15.278 ********* 2021-03-09T02:44:19.226 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-03-09T02:44:19.228 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:19.255 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.066) 0:01:15.345 ********* 2021-03-09T02:44:19.272 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:19.297 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.041) 0:01:15.387 ********* 2021-03-09T02:44:19.325 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-03-09T02:44:19.327 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:19.351 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.053) 0:01:15.441 ********* 2021-03-09T02:44:19.368 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:19.395 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.044) 0:01:15.485 ********* 2021-03-09T02:44:19.413 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-03-09T02:44:19.415 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-03-09T02:44:19.445 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.049) 0:01:15.535 ********* 2021-03-09T02:44:19.475 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2021-03-09T02:44:19.502 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.057) 0:01:15.592 ********* 2021-03-09T02:44:19.504 INFO:teuthology.task.ansible.out: 2021-03-09T02:44:19.521 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-03-09T02:44:19.523 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-03-09T02:44:19.548 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.045) 0:01:15.638 ********* 2021-03-09T02:44:19.597 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "no_recommended_packages": ["collectl"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2021-03-09T02:44:19.625 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.077) 0:01:15.715 ********* 2021-03-09T02:44:19.632 INFO:teuthology.task.ansible.out: 2021-03-09T02:44:19.634 INFO:teuthology.task.ansible.out:TASK [testnode : Including major version specific variables.] ****************** ok: [smithi092.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-03-09T02:44:19.660 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.034) 0:01:15.750 ********* 2021-03-09T02:44:19.661 INFO:teuthology.task.ansible.out: 2021-03-09T02:44:19.710 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2021-03-09T02:44:19.736 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.076) 0:01:15.826 ********* 2021-03-09T02:44:19.742 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-03-09T02:44:19.744 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-03-09T02:44:19.769 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.032) 0:01:15.859 ********* 2021-03-09T02:44:19.818 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-03-09T02:44:19.852 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.082) 0:01:15.942 ********* 2021-03-09T02:44:19.900 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-03-09T02:44:19.902 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-03-09T02:44:19.925 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:19 +0000 (0:00:00.073) 0:01:16.015 ********* 2021-03-09T02:44:19.985 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2021-03-09T02:44:20.010 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:20 +0000 (0:00:00.084) 0:01:16.100 ********* 2021-03-09T02:44:20.100 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-03-09T02:44:20.102 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-03-09T02:44:20.124 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:20 +0000 (0:00:00.114) 0:01:16.214 ********* 2021-03-09T02:44:20.193 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-03-09T02:44:20.195 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure the teuthology_user group exists.] ********************* ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "name": "ubuntu", "state": "present", "system": false} 2021-03-09T02:44:20.217 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:20 +0000 (0:00:00.093) 0:01:16.307 ********* 2021-03-09T02:44:20.240 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:20 +0000 (0:00:00.022) 0:01:16.330 ********* 2021-03-09T02:44:20.288 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-03-09T02:44:20.289 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:20.313 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:20 +0000 (0:00:00.073) 0:01:16.403 ********* 2021-03-09T02:44:20.320 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-03-09T02:44:20.321 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:20.346 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:20 +0000 (0:00:00.032) 0:01:16.436 ********* 2021-03-09T02:44:20.414 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-03-09T02:44:20.416 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:20.442 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:20 +0000 (0:00:00.096) 0:01:16.532 ********* 2021-03-09T02:44:20.533 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-03-09T02:44:20.535 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-03-09T02:44:20.559 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:20 +0000 (0:00:00.116) 0:01:16.649 ********* 2021-03-09T02:44:20.651 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2021-03-09T02:44:20.677 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:20 +0000 (0:00:00.116) 0:01:16.765 ********* 2021-03-09T02:44:21.017 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-03-09T02:44:21.019 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-03-09T02:44:21.044 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:21 +0000 (0:00:00.368) 0:01:17.134 ********* 2021-03-09T02:44:21.124 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-03-09T02:44:21.127 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi092"}, "changed": false} 2021-03-09T02:44:21.158 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:21 +0000 (0:00:00.114) 0:01:17.248 ********* 2021-03-09T02:44:21.165 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-03-09T02:44:21.167 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2021-03-09T02:44:21.191 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:21 +0000 (0:00:00.032) 0:01:17.281 ********* 2021-03-09T02:44:21.270 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-03-09T02:44:21.272 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi144"}, "changed": false} 2021-03-09T02:44:21.293 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:21 +0000 (0:00:00.102) 0:01:17.383 ********* 2021-03-09T02:44:22.011 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-03-09T02:44:22.013 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi092", "ansible_hostname": "smithi092", "ansible_nodename": "smithi092"}, "changed": false, "name": "smithi092"} 2021-03-09T02:44:22.035 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:22 +0000 (0:00:00.741) 0:01:18.125 ********* 2021-03-09T02:44:22.052 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "", "ansible_fqdn": "smithi144", "ansible_hostname": "smithi144", "ansible_nodename": "smithi144"}, "changed": false, "name": "smithi144"} 2021-03-09T02:44:22.077 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:22 +0000 (0:00:00.041) 0:01:18.167 ********* 2021-03-09T02:44:22.407 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-03-09T02:44:22.408 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-03-09T02:44:22.433 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:22 +0000 (0:00:00.356) 0:01:18.523 ********* 2021-03-09T02:44:22.454 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2021-03-09T02:44:22.484 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:22 +0000 (0:00:00.051) 0:01:18.574 ********* 2021-03-09T02:44:22.824 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-03-09T02:44:22.826 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-03-09T02:44:22.851 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:22 +0000 (0:00:00.367) 0:01:18.941 ********* 2021-03-09T02:44:22.869 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2021-03-09T02:44:22.895 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:22 +0000 (0:00:00.044) 0:01:18.985 ********* 2021-03-09T02:44:23.307 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-03-09T02:44:23.309 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-03-09T02:44:23.332 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.436) 0:01:19.422 ********* 2021-03-09T02:44:23.339 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-03-09T02:44:23.364 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.032) 0:01:19.454 ********* 2021-03-09T02:44:23.401 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-03-09T02:44:23.403 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.425 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.061) 0:01:19.515 ********* 2021-03-09T02:44:23.442 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.470 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.044) 0:01:19.560 ********* 2021-03-09T02:44:23.497 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-03-09T02:44:23.499 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.523 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.053) 0:01:19.613 ********* 2021-03-09T02:44:23.540 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.542 INFO:teuthology.task.ansible.out: 2021-03-09T02:44:23.565 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.041) 0:01:19.655 ********* 2021-03-09T02:44:23.593 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-03-09T02:44:23.595 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.618 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.052) 0:01:19.708 ********* 2021-03-09T02:44:23.635 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.659 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.041) 0:01:19.749 ********* 2021-03-09T02:44:23.687 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-03-09T02:44:23.688 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.712 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.052) 0:01:19.802 ********* 2021-03-09T02:44:23.740 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.768 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.056) 0:01:19.858 ********* 2021-03-09T02:44:23.785 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-03-09T02:44:23.787 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.811 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.042) 0:01:19.901 ********* 2021-03-09T02:44:23.838 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.863 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.051) 0:01:19.953 ********* 2021-03-09T02:44:23.879 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-03-09T02:44:23.881 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.904 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.041) 0:01:19.995 ********* 2021-03-09T02:44:23.932 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.956 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.051) 0:01:20.046 ********* 2021-03-09T02:44:23.972 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-03-09T02:44:23.974 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:23.997 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:23 +0000 (0:00:00.041) 0:01:20.087 ********* 2021-03-09T02:44:24.025 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.050 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.052) 0:01:20.140 ********* 2021-03-09T02:44:24.067 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-03-09T02:44:24.069 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.092 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.042) 0:01:20.182 ********* 2021-03-09T02:44:24.121 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.145 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.052) 0:01:20.235 ********* 2021-03-09T02:44:24.162 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-03-09T02:44:24.187 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.041) 0:01:20.277 ********* 2021-03-09T02:44:24.238 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.050) 0:01:20.328 ********* 2021-03-09T02:44:24.256 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-03-09T02:44:24.280 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.042) 0:01:20.371 ********* 2021-03-09T02:44:24.336 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.055) 0:01:20.426 ********* 2021-03-09T02:44:24.353 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-03-09T02:44:24.378 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.042) 0:01:20.468 ********* 2021-03-09T02:44:24.420 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.042) 0:01:20.510 ********* 2021-03-09T02:44:24.447 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-03-09T02:44:24.448 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.472 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.051) 0:01:20.562 ********* 2021-03-09T02:44:24.489 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.514 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.042) 0:01:20.604 ********* 2021-03-09T02:44:24.552 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-03-09T02:44:24.553 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.578 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.063) 0:01:20.668 ********* 2021-03-09T02:44:24.595 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.597 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.620 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.042) 0:01:20.710 ********* 2021-03-09T02:44:24.647 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-03-09T02:44:24.649 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.672 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.051) 0:01:20.762 ********* 2021-03-09T02:44:24.688 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.712 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.040) 0:01:20.802 ********* 2021-03-09T02:44:24.739 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-09T02:44:24.741 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.764 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.052) 0:01:20.855 ********* 2021-03-09T02:44:24.782 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.806 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.041) 0:01:20.896 ********* 2021-03-09T02:44:24.833 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-03-09T02:44:24.835 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.858 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.051) 0:01:20.948 ********* 2021-03-09T02:44:24.875 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.904 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.046) 0:01:20.994 ********* 2021-03-09T02:44:24.931 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-03-09T02:44:24.933 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.956 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.051) 0:01:21.046 ********* 2021-03-09T02:44:24.973 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:24.998 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:24 +0000 (0:00:00.042) 0:01:21.089 ********* 2021-03-09T02:44:25.017 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-03-09T02:44:25.018 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.042 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.043) 0:01:21.132 ********* 2021-03-09T02:44:25.070 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.096 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.053) 0:01:21.186 ********* 2021-03-09T02:44:25.112 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-03-09T02:44:25.114 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.139 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.043) 0:01:21.229 ********* 2021-03-09T02:44:25.167 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.192 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.052) 0:01:21.282 ********* 2021-03-09T02:44:25.218 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-03-09T02:44:25.220 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.244 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.052) 0:01:21.334 ********* 2021-03-09T02:44:25.262 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.289 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.045) 0:01:21.379 ********* 2021-03-09T02:44:25.317 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-03-09T02:44:25.319 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.344 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.054) 0:01:21.434 ********* 2021-03-09T02:44:25.361 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.386 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.042) 0:01:21.476 ********* 2021-03-09T02:44:25.425 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-03-09T02:44:25.426 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.449 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.063) 0:01:21.540 ********* 2021-03-09T02:44:25.467 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.497 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.047) 0:01:21.587 ********* 2021-03-09T02:44:25.536 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-03-09T02:44:25.538 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.561 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.064) 0:01:21.651 ********* 2021-03-09T02:44:25.567 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.593 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.032) 0:01:21.683 ********* 2021-03-09T02:44:25.641 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-03-09T02:44:25.643 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.668 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.074) 0:01:21.758 ********* 2021-03-09T02:44:25.675 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.703 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.035) 0:01:21.793 ********* 2021-03-09T02:44:25.732 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.758 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.054) 0:01:21.848 ********* 2021-03-09T02:44:25.775 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.800 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.042) 0:01:21.890 ********* 2021-03-09T02:44:25.838 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2021-03-09T02:44:25.840 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.864 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.064) 0:01:21.954 ********* 2021-03-09T02:44:25.881 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.907 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.042) 0:01:21.997 ********* 2021-03-09T02:44:25.945 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-03-09T02:44:25.947 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:25.971 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:25 +0000 (0:00:00.064) 0:01:22.061 ********* 2021-03-09T02:44:25.981 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.006 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.035) 0:01:22.096 ********* 2021-03-09T02:44:26.054 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-03-09T02:44:26.056 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.079 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.072) 0:01:22.169 ********* 2021-03-09T02:44:26.086 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.116 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.036) 0:01:22.205 ********* 2021-03-09T02:44:26.153 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-03-09T02:44:26.154 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.177 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.061) 0:01:22.267 ********* 2021-03-09T02:44:26.184 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.210 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.032) 0:01:22.300 ********* 2021-03-09T02:44:26.237 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.263 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.052) 0:01:22.352 ********* 2021-03-09T02:44:26.301 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.326 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.063) 0:01:22.416 ********* 2021-03-09T02:44:26.343 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-03-09T02:44:26.344 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.368 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.042) 0:01:22.458 ********* 2021-03-09T02:44:26.396 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.424 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.055) 0:01:22.514 ********* 2021-03-09T02:44:26.440 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-03-09T02:44:26.442 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.470 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.046) 0:01:22.560 ********* 2021-03-09T02:44:26.541 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.543 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.570 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.099) 0:01:22.660 ********* 2021-03-09T02:44:26.590 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.020) 0:01:22.680 ********* 2021-03-09T02:44:26.671 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:26.696 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.105) 0:01:22.786 ********* 2021-03-09T02:44:26.744 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-03-09T02:44:26.746 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005134", "end": "2021-03-09 02:44:26.273814", "rc": 0, "start": "2021-03-09 02:44:26.268680", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:44:26.770 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.073) 0:01:22.860 ********* 2021-03-09T02:44:26.850 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-03-09T02:44:26.878 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.107) 0:01:22.968 ********* 2021-03-09T02:44:26.895 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-03-09T02:44:26.896 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.005205", "end": "2021-03-09 02:44:26.631459", "rc": 0, "start": "2021-03-09 02:44:26.626254", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:44:26.920 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:26 +0000 (0:00:00.041) 0:01:23.010 ********* 2021-03-09T02:44:26.988 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-03-09T02:44:27.012 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:27 +0000 (0:00:00.092) 0:01:23.102 ********* 2021-03-09T02:44:27.248 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-03-09T02:44:27.250 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-03-09T02:44:27.278 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:27 +0000 (0:00:00.266) 0:01:23.368 ********* 2021-03-09T02:44:27.347 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-03-09T02:44:27.349 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:27.373 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:27 +0000 (0:00:00.094) 0:01:23.463 ********* 2021-03-09T02:44:27.401 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-03-09T02:44:27.402 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2021-03-09T02:44:27.429 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:27 +0000 (0:00:00.056) 0:01:23.519 ********* 2021-03-09T02:44:27.511 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:27.534 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:27 +0000 (0:00:00.105) 0:01:23.624 ********* 2021-03-09T02:44:28.705 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-03-09T02:44:28.707 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} ok: [smithi144.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2021-03-09T02:44:29.640 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-03-09T02:44:29.641 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2021-03-09T02:44:29.665 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:29 +0000 (0:00:02.131) 0:01:25.755 ********* 2021-03-09T02:44:29.689 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:29 +0000 (0:00:00.023) 0:01:25.779 ********* 2021-03-09T02:44:30.631 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-03-09T02:44:30.633 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:44:30.655 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:30 +0000 (0:00:00.965) 0:01:26.745 ********* 2021-03-09T02:44:30.679 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:30 +0000 (0:00:00.023) 0:01:26.769 ********* 2021-03-09T02:44:30.717 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-03-09T02:44:30.743 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:30 +0000 (0:00:00.064) 0:01:26.833 ********* 2021-03-09T02:44:30.785 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:30 +0000 (0:00:00.042) 0:01:26.875 ********* 2021-03-09T02:44:30.958 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-03-09T02:44:30.960 INFO:teuthology.task.ansible.out:[WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.036738", "end": "2021-03-09 02:44:30.482953", "rc": 0, "start": "2021-03-09 02:44:30.446215", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:44:30.985 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:30 +0000 (0:00:00.200) 0:01:27.076 ********* 2021-03-09T02:44:31.004 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.037098", "end": "2021-03-09 02:44:30.742720", "rc": 0, "start": "2021-03-09 02:44:30.705622", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:44:31.005 INFO:teuthology.task.ansible.out: 2021-03-09T02:44:31.032 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:31 +0000 (0:00:00.046) 0:01:27.123 ********* 2021-03-09T02:44:33.964 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-03-09T02:44:33.965 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:44:33.990 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:33 +0000 (0:00:02.957) 0:01:30.080 ********* 2021-03-09T02:44:34.049 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:44:34.077 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:34 +0000 (0:00:00.086) 0:01:30.167 ********* 2021-03-09T02:44:44.086 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-03-09T02:44:44.088 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 15 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102064 files and directorie 2021-03-09T02:44:44.090 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 15 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Read 2021-03-09T02:44:44.092 INFO:teuthology.task.ansible.out:ing database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102064 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-03-09T02:44:44.113 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:44 +0000 (0:00:10.035) 0:01:40.203 ********* 2021-03-09T02:44:44.215 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 15 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 102064 files and directorie 2021-03-09T02:44:44.218 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 15 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Read 2021-03-09T02:44:44.220 INFO:teuthology.task.ansible.out:ing database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 102064 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-03-09T02:44:44.242 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:44 +0000 (0:00:00.129) 0:01:40.332 ********* 2021-03-09T02:44:45.077 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-03-09T02:44:45.079 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:44:45.102 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:45 +0000 (0:00:00.859) 0:01:41.192 ********* 2021-03-09T02:44:45.103 INFO:teuthology.task.ansible.out: 2021-03-09T02:44:45.155 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:44:45.195 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:45 +0000 (0:00:00.088) 0:01:41.280 ********* 2021-03-09T02:44:45.200 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-03-09T02:44:45.202 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:45.233 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:45 +0000 (0:00:00.043) 0:01:41.323 ********* 2021-03-09T02:44:45.273 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:45.311 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:45 +0000 (0:00:00.078) 0:01:41.401 ********* 2021-03-09T02:44:54.022 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-03-09T02:44:54.024 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 15 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.36 [870 kB]\nFetched 10.4 MB in 1s (9744 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database .. 2021-03-09T02:44:54.025 INFO:teuthology.task.ansible.out:. 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101922 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubunt 2021-03-09T02:44:54.027 INFO:teuthology.task.ansible.out:u1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 15 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.36 [870 kB]", "Fetched 10.4 MB in 1s (9744 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(R 2021-03-09T02:44:54.029 INFO:teuthology.task.ansible.out:eading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101922 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-03-09T02:44:54.050 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:54 +0000 (0:00:08.738) 0:01:50.140 ********* 2021-03-09T02:44:54.071 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 15 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.36 [870 kB]\nFetched 10.4 MB in 1s (9680 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database .. 2021-03-09T02:44:54.073 INFO:teuthology.task.ansible.out:. 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 101922 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubunt 2021-03-09T02:44:54.075 INFO:teuthology.task.ansible.out:u1.4) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 15 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.6 [2724 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.6 [925 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.36 [40.5 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.36 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.36 [5176 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.36 [870 kB]", "Fetched 10.4 MB in 1s (9680 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(R 2021-03-09T02:44:54.077 INFO:teuthology.task.ansible.out:eading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 101922 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.6_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.36_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.6) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.36) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.36) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.4) ..."]} 2021-03-09T02:44:54.099 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:54 +0000 (0:00:00.049) 0:01:50.189 ********* 2021-03-09T02:44:54.966 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-03-09T02:44:54.968 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:44:54.991 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:54 +0000 (0:00:00.891) 0:01:51.081 ********* 2021-03-09T02:44:55.019 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:44:55.046 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:55 +0000 (0:00:00.054) 0:01:51.136 ********* 2021-03-09T02:44:55.932 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-03-09T02:44:55.934 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:44:55.958 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:55 +0000 (0:00:00.911) 0:01:52.047 ********* 2021-03-09T02:44:55.978 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:55 +0000 (0:00:00.021) 0:01:52.068 ********* 2021-03-09T02:44:56.131 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-03-09T02:44:56.133 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-03-09T02:44:56.157 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:56 +0000 (0:00:00.178) 0:01:52.247 ********* 2021-03-09T02:44:56.164 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2021-03-09T02:44:56.189 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:56 +0000 (0:00:00.032) 0:01:52.279 ********* 2021-03-09T02:44:56.238 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:56.263 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:56 +0000 (0:00:00.073) 0:01:52.353 ********* 2021-03-09T02:44:56.269 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:56.298 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:56 +0000 (0:00:00.034) 0:01:52.388 ********* 2021-03-09T02:44:56.440 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-03-09T02:44:56.442 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-03-09T02:44:56.465 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:56 +0000 (0:00:00.167) 0:01:52.555 ********* 2021-03-09T02:44:56.483 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2021-03-09T02:44:56.509 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:56 +0000 (0:00:00.044) 0:01:52.599 ********* 2021-03-09T02:44:56.840 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-03-09T02:44:56.841 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-03-09T02:44:56.864 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:56 +0000 (0:00:00.354) 0:01:52.954 ********* 2021-03-09T02:44:56.884 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2021-03-09T02:44:56.911 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:56 +0000 (0:00:00.046) 0:01:53.001 ********* 2021-03-09T02:44:57.086 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-03-09T02:44:57.087 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-03-09T02:44:57.114 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:57 +0000 (0:00:00.202) 0:01:53.204 ********* 2021-03-09T02:44:57.122 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1238, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-03-09T02:44:57.151 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:57 +0000 (0:00:00.037) 0:01:53.241 ********* 2021-03-09T02:44:57.318 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2021-03-09T02:44:57.320 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-09T02:44:57.342 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:57 +0000 (0:00:00.191) 0:01:53.432 ********* 2021-03-09T02:44:57.350 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-09T02:44:57.379 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:57 +0000 (0:00:00.036) 0:01:53.469 ********* 2021-03-09T02:44:57.685 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2021-03-09T02:44:57.686 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Tue 2021-03-09 02:42:08 UTC", "ActiveEnterTimestampMonotonic": "17166277", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target nss-lookup.target system.slice -.mount systemd-tmpfiles-setup.service remote-fs.target network.target sysinit.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2021-03-09 02:42:08 UTC", "AssertTimestampMonotonic": "16667479", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2021-03-09 02:42:08 UTC", "ConditionTimestampMonotonic": "16667477", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-03-09T02:44:57.687 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1383", "ExecMainStartTimestamp": "Tue 2021-03-09 02:42:08 UTC", "ExecMainStartTimestampMonotonic": "17166266", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2021-03-09 02:42:08 UTC", "InactiveExitTimestampMonotonic": "16668169", "InvocationID": "7cb052908b4d423c8c839073a5879475", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-03-09T02:44:57.689 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1383", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-03-09T02:44:57.690 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2021-03-09 02:42:08 UTC", "StateChangeTimestampMonotonic": "17166277", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2021-03-09 02:42:08 UTC", "WatchdogTimestampMonotonic": "17166276", "WatchdogUSec": "0"}} 2021-03-09T02:44:57.711 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:57 +0000 (0:00:00.332) 0:01:53.801 ********* 2021-03-09T02:44:57.720 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "name": "apache2", "state": "stopped", "status": {"ActiveEnterTimestamp": "Tue 2021-03-09 02:42:14 UTC", "ActiveEnterTimestampMonotonic": "17937444", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "remote-fs.target systemd-journald.socket sysinit.target system.slice nss-lookup.target basic.target -.mount systemd-tmpfiles-setup.service network.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2021-03-09 02:42:14 UTC", "AssertTimestampMonotonic": "17477221", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2021-03-09 02:42:14 UTC", "ConditionTimestampMonotonic": "17477221", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ControlGroup": "/system.slice/apache2.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "The Apache HTTP Server", "DevicePolicy": "auto", " 2021-03-09T02:44:57.722 INFO:teuthology.task.ansible.out:DropInPaths": "/lib/systemd/system/apache2.service.d/apache2-systemd.conf", "DynamicUser": "no", "Environment": "APACHE_STARTED_BY_SYSTEMD=true", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1255", "ExecMainStartTimestamp": "Tue 2021-03-09 02:42:14 UTC", "ExecMainStartTimestampMonotonic": "17937426", "ExecMainStatus": "0", "ExecReload": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl graceful ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl start ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStop": "{ path=/usr/sbin/apachectl ; argv[]=/usr/sbin/apachectl stop ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/apache2.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "apache2.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2021-03-09 02:42:14 UTC", "InactiveExitTimestampMonotonic": "17477955", "InvocationID": "9195c239544747a29ccfcf835077c3d2", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESof 2021-03-09T02:44:57.724 INFO:teuthology.task.ansible.out:t": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1255", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "apache2.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "sysinit.target -.mount system.slice", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "on-abort", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIG 2021-03-09T02:44:57.726 INFO:teuthology.task.ansible.out:HUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2021-03-09 02:42:14 UTC", "StateChangeTimestampMonotonic": "17937444", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "55", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2021-03-09 02:42:14 UTC", "WatchdogTimestampMonotonic": "17937443", "WatchdogUSec": "0"}} 2021-03-09T02:44:57.752 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:57 +0000 (0:00:00.040) 0:01:53.842 ********* 2021-03-09T02:44:57.790 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:57.814 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:57 +0000 (0:00:00.062) 0:01:53.904 ********* 2021-03-09T02:44:57.821 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:57.848 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:57 +0000 (0:00:00.034) 0:01:53.938 ********* 2021-03-09T02:44:57.888 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:57.914 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:57 +0000 (0:00:00.065) 0:01:54.004 ********* 2021-03-09T02:44:57.921 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:57.950 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:57 +0000 (0:00:00.035) 0:01:54.040 ********* 2021-03-09T02:44:57.988 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-03-09T02:44:57.990 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.013 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.063) 0:01:54.103 ********* 2021-03-09T02:44:58.031 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.057 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.043) 0:01:54.147 ********* 2021-03-09T02:44:58.095 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-03-09T02:44:58.097 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.119 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.061) 0:01:54.209 ********* 2021-03-09T02:44:58.137 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.167 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.048) 0:01:54.257 ********* 2021-03-09T02:44:58.185 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-03-09T02:44:58.186 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.210 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.042) 0:01:54.300 ********* 2021-03-09T02:44:58.259 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.284 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.074) 0:01:54.374 ********* 2021-03-09T02:44:58.291 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.316 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.031) 0:01:54.406 ********* 2021-03-09T02:44:58.364 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.392 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.076) 0:01:54.482 ********* 2021-03-09T02:44:58.399 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-03-09T02:44:58.400 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.426 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.033) 0:01:54.516 ********* 2021-03-09T02:44:58.453 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.478 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.052) 0:01:54.569 ********* 2021-03-09T02:44:58.495 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-03-09T02:44:58.497 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.521 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.042) 0:01:54.611 ********* 2021-03-09T02:44:58.538 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.563 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.042) 0:01:54.653 ********* 2021-03-09T02:44:58.590 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-09T02:44:58.592 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.615 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.052) 0:01:54.705 ********* 2021-03-09T02:44:58.643 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.668 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.052) 0:01:54.758 ********* 2021-03-09T02:44:58.716 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-03-09T02:44:58.718 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.740 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.071) 0:01:54.830 ********* 2021-03-09T02:44:58.747 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.773 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.032) 0:01:54.863 ********* 2021-03-09T02:44:58.811 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-03-09T02:44:58.813 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.837 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.064) 0:01:54.927 ********* 2021-03-09T02:44:58.855 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.879 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.041) 0:01:54.969 ********* 2021-03-09T02:44:58.906 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-03-09T02:44:58.907 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.930 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.051) 0:01:55.020 ********* 2021-03-09T02:44:58.959 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:58.990 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:58 +0000 (0:00:00.059) 0:01:55.080 ********* 2021-03-09T02:44:59.007 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-03-09T02:44:59.008 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.032 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.041) 0:01:55.122 ********* 2021-03-09T02:44:59.059 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.086 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.054) 0:01:55.176 ********* 2021-03-09T02:44:59.114 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-09T02:44:59.116 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.139 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.052) 0:01:55.229 ********* 2021-03-09T02:44:59.157 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.183 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.044) 0:01:55.274 ********* 2021-03-09T02:44:59.221 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-09T02:44:59.222 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.249 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.065) 0:01:55.339 ********* 2021-03-09T02:44:59.256 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.282 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.032) 0:01:55.372 ********* 2021-03-09T02:44:59.320 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-03-09T02:44:59.321 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.346 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.064) 0:01:55.436 ********* 2021-03-09T02:44:59.353 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.378 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.032) 0:01:55.468 ********* 2021-03-09T02:44:59.416 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-03-09T02:44:59.418 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.441 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.062) 0:01:55.531 ********* 2021-03-09T02:44:59.459 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.484 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.042) 0:01:55.574 ********* 2021-03-09T02:44:59.511 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-03-09T02:44:59.513 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.536 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.052) 0:01:55.626 ********* 2021-03-09T02:44:59.564 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.592 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.055) 0:01:55.682 ********* 2021-03-09T02:44:59.609 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-03-09T02:44:59.611 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.635 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.043) 0:01:55.726 ********* 2021-03-09T02:44:59.674 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.699 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.063) 0:01:55.789 ********* 2021-03-09T02:44:59.705 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-09T02:44:59.707 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.732 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.033) 0:01:55.822 ********* 2021-03-09T02:44:59.772 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.797 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.065) 0:01:55.887 ********* 2021-03-09T02:44:59.807 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-09T02:44:59.808 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.833 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.035) 0:01:55.923 ********* 2021-03-09T02:44:59.871 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.897 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.063) 0:01:55.987 ********* 2021-03-09T02:44:59.913 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-03-09T02:44:59.915 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.938 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.041) 0:01:56.028 ********* 2021-03-09T02:44:59.966 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:44:59.992 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:44:59 +0000 (0:00:00.053) 0:01:56.082 ********* 2021-03-09T02:45:00.019 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-03-09T02:45:00.021 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:00.043 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:00 +0000 (0:00:00.051) 0:01:56.134 ********* 2021-03-09T02:45:00.061 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:00.086 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:00 +0000 (0:00:00.042) 0:01:56.176 ********* 2021-03-09T02:45:00.123 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-03-09T02:45:00.125 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:00.148 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:00 +0000 (0:00:00.062) 0:01:56.238 ********* 2021-03-09T02:45:00.166 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:00.194 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:00 +0000 (0:00:00.045) 0:01:56.284 ********* 2021-03-09T02:45:00.222 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-03-09T02:45:00.224 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:00.247 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:00 +0000 (0:00:00.053) 0:01:56.337 ********* 2021-03-09T02:45:00.266 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:00.291 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:00 +0000 (0:00:00.043) 0:01:56.381 ********* 2021-03-09T02:45:00.320 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-03-09T02:45:00.322 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:00.354 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:00 +0000 (0:00:00.063) 0:01:56.444 ********* 2021-03-09T02:45:00.372 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:00.405 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:00 +0000 (0:00:00.050) 0:01:56.495 ********* 2021-03-09T02:45:00.559 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-03-09T02:45:00.561 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-03-09T02:45:00.584 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:00 +0000 (0:00:00.179) 0:01:56.674 ********* 2021-03-09T02:45:00.604 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2021-03-09T02:45:00.631 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:00 +0000 (0:00:00.046) 0:01:56.721 ********* 2021-03-09T02:45:00.954 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-03-09T02:45:00.956 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-03-09T02:45:00.979 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:00 +0000 (0:00:00.348) 0:01:57.069 ********* 2021-03-09T02:45:01.006 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2021-03-09T02:45:01.032 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:01 +0000 (0:00:00.052) 0:01:57.121 ********* 2021-03-09T02:45:01.050 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-03-09T02:45:01.051 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:01.078 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:01 +0000 (0:00:00.047) 0:01:57.168 ********* 2021-03-09T02:45:01.107 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:01.131 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:01 +0000 (0:00:00.052) 0:01:57.221 ********* 2021-03-09T02:45:01.479 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2021-03-09T02:45:01.481 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-03-09T02:45:01.506 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:01 +0000 (0:00:00.374) 0:01:57.596 ********* 2021-03-09T02:45:01.524 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2021-03-09T02:45:01.550 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:01 +0000 (0:00:00.044) 0:01:57.640 ********* 2021-03-09T02:45:01.874 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-03-09T02:45:01.900 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:01 +0000 (0:00:00.348) 0:01:57.989 ********* 2021-03-09T02:45:01.927 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2021-03-09T02:45:01.953 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:01 +0000 (0:00:00.054) 0:01:58.043 ********* 2021-03-09T02:45:02.288 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-03-09T02:45:02.290 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-03-09T02:45:02.312 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:02 +0000 (0:00:00.358) 0:01:58.402 ********* 2021-03-09T02:45:02.341 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2021-03-09T02:45:02.370 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:02 +0000 (0:00:00.058) 0:01:58.461 ********* 2021-03-09T02:45:02.503 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-03-09T02:45:02.505 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-09T02:45:02.529 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:02 +0000 (0:00:00.158) 0:01:58.619 ********* 2021-03-09T02:45:02.567 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-09T02:45:02.594 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:02 +0000 (0:00:00.064) 0:01:58.684 ********* 2021-03-09T02:45:02.602 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-03-09T02:45:02.603 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:02.628 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:02 +0000 (0:00:00.033) 0:01:58.718 ********* 2021-03-09T02:45:02.666 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:02.693 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:02 +0000 (0:00:00.065) 0:01:58.783 ********* 2021-03-09T02:45:02.710 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-03-09T02:45:02.712 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:02.735 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:02 +0000 (0:00:00.041) 0:01:58.825 ********* 2021-03-09T02:45:02.762 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:02.787 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:02 +0000 (0:00:00.052) 0:01:58.877 ********* 2021-03-09T02:45:02.804 INFO:teuthology.task.ansible.out: 2021-03-09T02:45:02.806 INFO:teuthology.task.ansible.out:TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:02.831 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:02 +0000 (0:00:00.044) 0:01:58.921 ********* 2021-03-09T02:45:02.859 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:02.883 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:02 +0000 (0:00:00.051) 0:01:58.973 ********* 2021-03-09T02:45:02.900 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-03-09T02:45:02.902 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:02.935 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:02 +0000 (0:00:00.052) 0:01:59.025 ********* 2021-03-09T02:45:02.986 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.015 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.079) 0:01:59.105 ********* 2021-03-09T02:45:03.142 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-03-09T02:45:03.144 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.168 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.152) 0:01:59.258 ********* 2021-03-09T02:45:03.207 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.233 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.064) 0:01:59.323 ********* 2021-03-09T02:45:03.253 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-03-09T02:45:03.276 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.043) 0:01:59.366 ********* 2021-03-09T02:45:03.314 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-03-09T02:45:03.339 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.063) 0:01:59.430 ********* 2021-03-09T02:45:03.366 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-03-09T02:45:03.368 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.379 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.380 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.390 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.402 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-03-09T02:45:03.404 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.415 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.416 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.426 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.428 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.438 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.440 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.468 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.128) 0:01:59.558 ********* 2021-03-09T02:45:03.475 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.476 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.476 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-03-09T02:45:03.477 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.477 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.488 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.490 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.500 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.502 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.511 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.542 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.074) 0:01:59.632 ********* 2021-03-09T02:45:03.717 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-03-09T02:45:03.719 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003607", "end": "2021-03-09 02:45:03.682018", "rc": 0, "start": "2021-03-09 02:45:03.678411", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/bin/teuthology-lock"]} 2021-03-09T02:45:03.744 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.201) 0:01:59.834 ********* 2021-03-09T02:45:03.772 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003485", "end": "2021-03-09 02:45:03.750449", "rc": 0, "start": "2021-03-09 02:45:03.746964", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_3082387bba74fcd24c9700593d10418152d53c97/virtualenv/bin/teuthology-lock"]} 2021-03-09T02:45:03.799 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.055) 0:01:59.889 ********* 2021-03-09T02:45:03.816 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-03-09T02:45:03.818 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.845 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.045) 0:01:59.935 ********* 2021-03-09T02:45:03.873 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.900 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.055) 0:01:59.990 ********* 2021-03-09T02:45:03.917 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-03-09T02:45:03.919 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:03.949 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.048) 0:02:00.039 ********* 2021-03-09T02:45:03.977 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:04.004 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:03 +0000 (0:00:00.055) 0:02:00.094 ********* 2021-03-09T02:45:04.147 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-03-09T02:45:04.149 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:04.171 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:04 +0000 (0:00:00.166) 0:02:00.261 ********* 2021-03-09T02:45:04.220 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:04.247 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:04 +0000 (0:00:00.075) 0:02:00.337 ********* 2021-03-09T02:45:05.230 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-03-09T02:45:05.232 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:45:05.257 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:05 +0000 (0:00:01.010) 0:02:01.347 ********* 2021-03-09T02:45:05.296 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:45:05.427 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:05 +0000 (0:00:00.169) 0:02:01.517 ********* 2021-03-09T02:45:05.443 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-03-09T02:45:05.445 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:05.473 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:05 +0000 (0:00:00.046) 0:02:01.563 ********* 2021-03-09T02:45:05.492 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:05.517 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:05 +0000 (0:00:00.044) 0:02:01.607 ********* 2021-03-09T02:45:05.555 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-03-09T02:45:05.556 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item={'block_used': 13643750, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451358, 'size_available': 927544762368, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451358, "block_size": 4096, "block_total": 240095108, "block_used": 13643750, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544762368, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}} 2021-03-09T02:45:05.582 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:05 +0000 (0:00:00.064) 0:02:01.672 ********* 2021-03-09T02:45:05.609 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item={'block_used': 13643749, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451359, 'size_available': 927544766464, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451359, "block_size": 4096, "block_total": 240095108, "block_used": 13643749, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544766464, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}} 2021-03-09T02:45:05.636 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:05 +0000 (0:00:00.054) 0:02:01.726 ********* 2021-03-09T02:45:05.778 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-03-09T02:45:05.780 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.010422", "end": "2021-03-09 02:45:05.306166", "rc": 0, "start": "2021-03-09 02:45:05.295744", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:05.806 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:05 +0000 (0:00:00.169) 0:02:01.896 ********* 2021-03-09T02:45:05.824 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.009684", "end": "2021-03-09 02:45:05.552988", "rc": 0, "start": "2021-03-09 02:45:05.543304", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:05.850 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:05 +0000 (0:00:00.043) 0:02:01.940 ********* 2021-03-09T02:45:05.887 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-03-09T02:45:05.889 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'block_used': 13643750, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451358, 'size_available': 927544762368, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451358, "block_size": 4096, "block_total": 240095108, "block_used": 13643750, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544762368, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:05.913 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:05 +0000 (0:00:00.063) 0:02:02.003 ********* 2021-03-09T02:45:05.941 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'block_used': 13643749, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451359, 'size_available': 927544766464, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451359, "block_size": 4096, "block_total": 240095108, "block_used": 13643749, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544766464, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:05.967 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:05 +0000 (0:00:00.053) 0:02:02.057 ********* 2021-03-09T02:45:05.983 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-03-09T02:45:06.008 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:06 +0000 (0:00:00.041) 0:02:02.098 ********* 2021-03-09T02:45:06.073 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:06 +0000 (0:00:00.064) 0:02:02.162 ********* 2021-03-09T02:45:06.101 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-03-09T02:45:06.103 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:06.165 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:07.899 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41q1XHhL1SpxjUhO7zNfBHtoCOTIjsyccw8'], 'uuids': ['ac6ad39b-8434-4c05-8a74-790f7eee5d61']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.606432", "end": "2021-03-09 02:45:07.628607", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41q1XHhL1SpxjUhO7zNfBHtoCOTIjsyccw8"], "labels": [], "masters": [], "uuids": ["ac6ad39b-8434-4c05-8a74-790f7eee5d61"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:06.022175", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-09T02:45:07.911 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:07.922 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:08.060 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOHxX3PlnLyCDEzRzAzSGUsaufMKGKZwjR'], 'uuids': ['e21ce18b-acbd-4b8e-97c4-a18879a9fe6d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.834599", "end": "2021-03-09 02:45:07.579144", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOHxX3PlnLyCDEzRzAzSGUsaufMKGKZwjR"], "labels": [], "masters": [], "uuids": ["e21ce18b-acbd-4b8e-97c4-a18879a9fe6d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:05.744545", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-09T02:45:08.072 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:08.083 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:09.202 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qOULsfxBiePg1F7HwKhxUkvgWsXcn5RIJ'], 'uuids': ['3010ffd4-c7ba-402a-a9e4-134bafd65552']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.156534", "end": "2021-03-09 02:45:08.928622", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qOULsfxBiePg1F7HwKhxUkvgWsXcn5RIJ"], "labels": [], "masters": [], "uuids": ["3010ffd4-c7ba-402a-a9e4-134bafd65552"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:07.772088", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-09T02:45:09.205 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XKPY', 'wwn-0x5000c5009293d715'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XKPY-part1', 'wwn-0x5000c5009293d715-part1'], 'uuids': ['c2492a7c-537b-4f45-893c-50f733187038']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293d715', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XKPY', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XKPY", "wwn-0x5000c5009293d715"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XKPY-part1", "wwn-0x5000c5009293d715-part1"], "labels": [], "masters": [], "uuids": ["c2492a7c-537b-4f45-893c-50f733187038"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XKPY", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293d715"}}, "skip_reason": 2021-03-09T02:45:09.206 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-03-09T02:45:09.322 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiO2dhi6equnL3Gs17f8OM50T3RnAYN2NHs'], 'uuids': ['7decd158-6698-433a-a912-0343fb3d74b3']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.110404", "end": "2021-03-09 02:45:08.835277", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiO2dhi6equnL3Gs17f8OM50T3RnAYN2NHs"], "labels": [], "masters": [], "uuids": ["7decd158-6698-433a-a912-0343fb3d74b3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:07.724873", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-09T02:45:09.323 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5LJ4S', 'wwn-0x5000c500918da5aa'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5LJ4S-part1', 'wwn-0x5000c500918da5aa-part1'], 'uuids': ['c2492a7c-537b-4f45-893c-50f733187038']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500918da5aa', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5LJ4S', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5LJ4S", "wwn-0x5000c500918da5aa"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5LJ4S-part1", "wwn-0x5000c500918da5aa-part1"], "labels": [], "masters": [], "uuids": ["c2492a7c-537b-4f45-893c-50f733187038"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5LJ4S", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500918da5aa"}}, "skip_reason": 2021-03-09T02:45:09.325 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-03-09T02:45:10.554 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-1aDXF7-z0jf-mq7e-ljs6-8zha-Geuu-j28xJP', 'nvme-INTEL_SSDPEDMD400G4_PHFT62050085400BGN', 'nvme-nvme.8086-50484654363230353030383534303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.206365", "end": "2021-03-09 02:45:10.284311", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-1aDXF7-z0jf-mq7e-ljs6-8zha-Geuu-j28xJP", "nvme-INTEL_SSDPEDMD400G4_PHFT62050085400BGN", "nvme-nvme.8086-50484654363230353030383534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:09.077946", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-03-09T02:45:10.556 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-09T02:45:10.566 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:10.578 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:10.912 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-ElZnEV-XYRM-wFml-uQ3l-vfal-nYqC-cieJi0', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400QM400BGN', 'nvme-nvme.8086-50484654363230343030514d34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.456437", "end": "2021-03-09 02:45:10.432650", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-ElZnEV-XYRM-wFml-uQ3l-vfal-nYqC-cieJi0", "nvme-INTEL_SSDPEDMD400G4_PHFT620400QM400BGN", "nvme-nvme.8086-50484654363230343030514d34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:08.976213", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-03-09T02:45:10.914 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:10.924 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:11.884 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qj5KJa8KvK2qnSJAEgghUdVSy2ECr33wA'], 'uuids': ['f6249641-9791-4bd3-8ef0-353e9b7f36b4']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.177992", "end": "2021-03-09 02:45:11.609304", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qj5KJa8KvK2qnSJAEgghUdVSy2ECr33wA"], "labels": [], "masters": [], "uuids": ["f6249641-9791-4bd3-8ef0-353e9b7f36b4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:10.431312", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-09T02:45:11.886 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:11.897 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:12.233 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiO68ZyRY9emXSfim0z3wD3Tx8kdwX8fHNY'], 'uuids': ['cb44ba11-5abd-44ee-b3bb-28854c657757']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.188739", "end": "2021-03-09 02:45:11.753709", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiO68ZyRY9emXSfim0z3wD3Tx8kdwX8fHNY"], "labels": [], "masters": [], "uuids": ["cb44ba11-5abd-44ee-b3bb-28854c657757"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:10.564970", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-09T02:45:12.245 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:12.256 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:13.182 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qqPN0iPt9AMvCLFYFSiwEbEDc1TA34FJv'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.156679", "end": "2021-03-09 02:45:12.909457", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qqPN0iPt9AMvCLFYFSiwEbEDc1TA34FJv"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:11.752778", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-09T02:45:13.194 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:13.508 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOt52cHbA8MCELiYuG1ksFzHIfZIWkacnp'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.132977", "end": "2021-03-09 02:45:13.028464", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOt52cHbA8MCELiYuG1ksFzHIfZIWkacnp"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:11.895487", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-09T02:45:13.520 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:14.462 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qbEjA3KHCu81Xkxb4yXSO8seXBNzTa999'], 'uuids': ['147ca02c-d8ad-4db9-a0f4-72ced096471e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.146401", "end": "2021-03-09 02:45:14.194129", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qbEjA3KHCu81Xkxb4yXSO8seXBNzTa999"], "labels": [], "masters": [], "uuids": ["147ca02c-d8ad-4db9-a0f4-72ced096471e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:13.047728", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-09T02:45:14.509 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:14 +0000 (0:00:08.436) 0:02:10.599 ********* 2021-03-09T02:45:14.611 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-09T02:45:14.613 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:14.759 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-03-09T02:45:14.761 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOf5jn7GDD2btuan7lr2Q6sTXlC83EweHl'], 'uuids': ['5e3e7217-c08a-4536-858e-cb303033c1f9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.107956", "end": "2021-03-09 02:45:14.278539", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOf5jn7GDD2btuan7lr2Q6sTXlC83EweHl"], "labels": [], "masters": [], "uuids": ["5e3e7217-c08a-4536-858e-cb303033c1f9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:13.170583", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-03-09T02:45:14.795 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:14 +0000 (0:00:00.285) 0:02:10.885 ********* 2021-03-09T02:45:14.885 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-03-09T02:45:14.887 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:14.908 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41q1XHhL1SpxjUhO7zNfBHtoCOTIjsyccw8'], 'uuids': ['ac6ad39b-8434-4c05-8a74-790f7eee5d61']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.157460", "end": "2021-03-09 02:45:14.632324", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41q1XHhL1SpxjUhO7zNfBHtoCOTIjsyccw8"], "labels": [], "masters": [], "uuids": ["ac6ad39b-8434-4c05-8a74-790f7eee5d61"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:14.474864", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152669 s, 756 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152669 s, 756 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:14.910 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:14.921 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:15.173 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOHxX3PlnLyCDEzRzAzSGUsaufMKGKZwjR'], 'uuids': ['e21ce18b-acbd-4b8e-97c4-a18879a9fe6d']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.164604", "end": "2021-03-09 02:45:14.698923", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOHxX3PlnLyCDEzRzAzSGUsaufMKGKZwjR"], "labels": [], "masters": [], "uuids": ["e21ce18b-acbd-4b8e-97c4-a18879a9fe6d"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:14.534319", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.16157 s, 714 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.16157 s, 714 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:15.184 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:15.196 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:15.207 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qOULsfxBiePg1F7HwKhxUkvgWsXcn5RIJ'], 'uuids': ['3010ffd4-c7ba-402a-a9e4-134bafd65552']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.160816", "end": "2021-03-09 02:45:14.939775", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qOULsfxBiePg1F7HwKhxUkvgWsXcn5RIJ"], "labels": [], "masters": [], "uuids": ["3010ffd4-c7ba-402a-a9e4-134bafd65552"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:14.778959", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.156153 s, 739 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.156153 s, 739 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:15.220 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XKPY', 'wwn-0x5000c5009293d715'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XKPY-part1', 'wwn-0x5000c5009293d715-part1'], 'uuids': ['c2492a7c-537b-4f45-893c-50f733187038']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c5009293d715', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XKPY', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XKPY", "wwn-0x5000c5009293d715"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XKPY-part1", "wwn-0x5000c5009293d715-part1"], "labels": [], "masters": [], "uuids": ["c2492a7c-537b-4f45-893c-50f733187038"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XKPY", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c5009293d715"}}, "skip_reason": 2021-03-09T02:45:15.221 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-03-09T02:45:15.398 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-1aDXF7-z0jf-mq7e-ljs6-8zha-Geuu-j28xJP', 'nvme-INTEL_SSDPEDMD400G4_PHFT62050085400BGN', 'nvme-nvme.8086-50484654363230353030383534303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.053856", "end": "2021-03-09 02:45:15.127089", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-1aDXF7-z0jf-mq7e-ljs6-8zha-Geuu-j28xJP", "nvme-INTEL_SSDPEDMD400G4_PHFT62050085400BGN", "nvme-nvme.8086-50484654363230353030383534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:15.073233", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0491374 s, 2.3 GB/s", "std 2021-03-09T02:45:15.400 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0491374 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:15.411 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:15.465 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiO2dhi6equnL3Gs17f8OM50T3RnAYN2NHs'], 'uuids': ['7decd158-6698-433a-a912-0343fb3d74b3']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.148514", "end": "2021-03-09 02:45:14.981529", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiO2dhi6equnL3Gs17f8OM50T3RnAYN2NHs"], "labels": [], "masters": [], "uuids": ["7decd158-6698-433a-a912-0343fb3d74b3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:14.833015", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.144398 s, 799 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.144398 s, 799 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:15.466 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'cfq', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5LJ4S', 'wwn-0x5000c500918da5aa'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5LJ4S-part1', 'wwn-0x5000c500918da5aa-part1'], 'uuids': ['c2492a7c-537b-4f45-893c-50f733187038']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c500918da5aa', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5LJ4S', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5LJ4S", "wwn-0x5000c500918da5aa"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5LJ4S-part1", "wwn-0x5000c500918da5aa-part1"], "labels": [], "masters": [], "uuids": ["c2492a7c-537b-4f45-893c-50f733187038"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "cfq", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5LJ4S", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500918da5aa"}}, "skip_reason": 2021-03-09T02:45:15.468 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2021-03-09T02:45:15.645 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-ElZnEV-XYRM-wFml-uQ3l-vfal-nYqC-cieJi0', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400QM400BGN', 'nvme-nvme.8086-50484654363230343030514d34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.049623", "end": "2021-03-09 02:45:15.156948", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-ElZnEV-XYRM-wFml-uQ3l-vfal-nYqC-cieJi0", "nvme-INTEL_SSDPEDMD400G4_PHFT620400QM400BGN", "nvme-nvme.8086-50484654363230343030514d34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:15.107325", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0449585 s, 2.6 GB/s", "std 2021-03-09T02:45:15.647 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0449585 s, 2.6 GB/s"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:15.657 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:15.668 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:15.701 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qj5KJa8KvK2qnSJAEgghUdVSy2ECr33wA'], 'uuids': ['f6249641-9791-4bd3-8ef0-353e9b7f36b4']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.157231", "end": "2021-03-09 02:45:15.421000", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qj5KJa8KvK2qnSJAEgghUdVSy2ECr33wA"], "labels": [], "masters": [], "uuids": ["f6249641-9791-4bd3-8ef0-353e9b7f36b4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:15.263769", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15274 s, 755 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15274 s, 755 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:15.702 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:15.714 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:15.956 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiO68ZyRY9emXSfim0z3wD3Tx8kdwX8fHNY'], 'uuids': ['cb44ba11-5abd-44ee-b3bb-28854c657757']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.157478", "end": "2021-03-09 02:45:15.472444", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiO68ZyRY9emXSfim0z3wD3Tx8kdwX8fHNY"], "labels": [], "masters": [], "uuids": ["cb44ba11-5abd-44ee-b3bb-28854c657757"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:15.314966", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.154472 s, 747 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.154472 s, 747 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:15.967 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:15.968 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:16.011 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qqPN0iPt9AMvCLFYFSiwEbEDc1TA34FJv'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.158995", "end": "2021-03-09 02:45:15.740809", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qqPN0iPt9AMvCLFYFSiwEbEDc1TA34FJv"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:15.581814", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.154246 s, 748 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.154246 s, 748 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:16.024 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:16.255 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOt52cHbA8MCELiYuG1ksFzHIfZIWkacnp'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.159569", "end": "2021-03-09 02:45:15.779957", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOt52cHbA8MCELiYuG1ksFzHIfZIWkacnp"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:15.620388", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.155628 s, 741 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.155628 s, 741 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:16.267 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'none', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:16.310 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qbEjA3KHCu81Xkxb4yXSO8seXBNzTa999'], 'uuids': ['147ca02c-d8ad-4db9-a0f4-72ced096471e']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.157158", "end": "2021-03-09 02:45:16.036744", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-uIhLChmnHbFNN9b3QcWZnCcSWKVKB41qbEjA3KHCu81Xkxb4yXSO8seXBNzTa999"], "labels": [], "masters": [], "uuids": ["147ca02c-d8ad-4db9-a0f4-72ced096471e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:15.879586", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.15299 s, 754 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.15299 s, 754 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:16.345 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:16 +0000 (0:00:01.550) 0:02:12.435 ********* 2021-03-09T02:45:16.553 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOf5jn7GDD2btuan7lr2Q6sTXlC83EweHl'], 'uuids': ['5e3e7217-c08a-4536-858e-cb303033c1f9']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.157720", "end": "2021-03-09 02:45:16.070177", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-JoGRYl14yPAFemLd8Hhhk81a3GfHXPiOf5jn7GDD2btuan7lr2Q6sTXlC83EweHl"], "labels": [], "masters": [], "uuids": ["5e3e7217-c08a-4536-858e-cb303033c1f9"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-03-09 02:45:15.912457", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.152941 s, 754 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.152941 s, 754 MB/s"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:16.591 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:16 +0000 (0:00:00.245) 0:02:12.681 ********* 2021-03-09T02:45:16.734 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-03-09T02:45:16.736 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.190105", "end": "2021-03-09 02:45:16.416330", "rc": 0, "start": "2021-03-09 02:45:16.226225", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:16.762 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:16 +0000 (0:00:00.171) 0:02:12.852 ********* 2021-03-09T02:45:16.938 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.146352", "end": "2021-03-09 02:45:16.404525", "rc": 0, "start": "2021-03-09 02:45:16.258173", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:16.962 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:16 +0000 (0:00:00.199) 0:02:13.052 ********* 2021-03-09T02:45:16.969 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-03-09T02:45:16.970 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.050711", "end": "2021-03-09 02:45:16.684634", "rc": 0, "start": "2021-03-09 02:45:16.633923", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:16.998 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:16 +0000 (0:00:00.035) 0:02:13.088 ********* 2021-03-09T02:45:17.070 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-03-09T02:45:17.138 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.140) 0:02:13.228 ********* 2021-03-09T02:45:17.165 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-03-09T02:45:17.167 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.194 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.055) 0:02:13.284 ********* 2021-03-09T02:45:17.221 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-03-09T02:45:17.223 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.246 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.052) 0:02:13.336 ********* 2021-03-09T02:45:17.274 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-03-09T02:45:17.276 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.304 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.057) 0:02:13.394 ********* 2021-03-09T02:45:17.331 INFO:teuthology.task.ansible.out: 2021-03-09T02:45:17.332 INFO:teuthology.task.ansible.out:TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.357 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.052) 0:02:13.447 ********* 2021-03-09T02:45:17.404 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-03-09T02:45:17.406 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.309876", "end": "2021-03-09 02:45:16.930120", "rc": 0, "start": "2021-03-09 02:45:16.620244", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:17.430 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.073) 0:02:13.520 ********* 2021-03-09T02:45:17.458 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-03-09T02:45:17.485 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.055) 0:02:13.576 ********* 2021-03-09T02:45:17.493 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-03-09T02:45:17.495 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-09T02:45:17.518 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.032) 0:02:13.608 ********* 2021-03-09T02:45:17.524 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-03-09T02:45:17.526 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.553 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.034) 0:02:13.643 ********* 2021-03-09T02:45:17.560 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-03-09T02:45:17.561 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'block_used': 13643749, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451359, 'size_available': 927544766464, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451359, "block_size": 4096, "block_total": 240095108, "block_used": 13643749, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544766464, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.589 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.036) 0:02:13.679 ********* 2021-03-09T02:45:17.596 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-03-09T02:45:17.598 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.622 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.033) 0:02:13.712 ********* 2021-03-09T02:45:17.629 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-03-09T02:45:17.631 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.656 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.033) 0:02:13.746 ********* 2021-03-09T02:45:17.787 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.035) 0:02:13.782 ********* TASK [testnode : set_fact] ***************************************************** skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.032) 0:02:13.814 ********* TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.031) 0:02:13.846 ********* TASK [testnode : set_fact] ***************************************************** skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.789 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.032) 0:02:13.878 ********* 2021-03-09T02:45:17.815 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-03-09T02:45:17.816 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.839 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.050) 0:02:13.929 ********* 2021-03-09T02:45:17.867 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-03-09T02:45:17.869 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.897 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.058) 0:02:13.987 ********* 2021-03-09T02:45:17.903 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-03-09T02:45:17.905 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-03-09T02:45:17.928 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.031) 0:02:14.018 ********* 2021-03-09T02:45:17.934 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-03-09T02:45:17.936 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.961 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.033) 0:02:14.051 ********* 2021-03-09T02:45:17.970 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-03-09T02:45:17.972 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'block_used': 13643750, 'uuid': 'c2492a7c-537b-4f45-893c-50f733187038', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226451358, 'size_available': 927544762368, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,errors=remount-ro,data=ordered', 'device': '/dev/sda1', 'inode_used': 113943, 'block_size': 4096, 'inode_available': 60941033}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226451358, "block_size": 4096, "block_total": 240095108, "block_used": 13643750, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60941033, "inode_total": 61054976, "inode_used": 113943, "mount": "/", "options": "rw,relatime,errors=remount-ro,data=ordered", "size_available": 927544762368, "size_total": 983429562368, "uuid": "c2492a7c-537b-4f45-893c-50f733187038"}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:17.997 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:17 +0000 (0:00:00.036) 0:02:14.087 ********* 2021-03-09T02:45:18.025 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-03-09T02:45:18.027 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:18.058 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:18 +0000 (0:00:00.060) 0:02:14.148 ********* 2021-03-09T02:45:18.088 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:18.120 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:18 +0000 (0:00:00.061) 0:02:14.210 ********* 2021-03-09T02:45:18.148 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-09T02:45:18.151 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:18.174 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:18 +0000 (0:00:00.054) 0:02:14.264 ********* 2021-03-09T02:45:18.201 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-03-09T02:45:18.203 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:18.225 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:18 +0000 (0:00:00.050) 0:02:14.315 ********* 2021-03-09T02:45:18.253 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:18.277 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:18 +0000 (0:00:00.052) 0:02:14.367 ********* 2021-03-09T02:45:18.315 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-03-09T02:45:18.317 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:18.340 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:18 +0000 (0:00:00.062) 0:02:14.430 ********* 2021-03-09T02:45:18.420 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-03-09T02:45:18.421 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-03-09T02:45:18.446 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:18 +0000 (0:00:00.105) 0:02:14.536 ********* 2021-03-09T02:45:18.686 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-03-09T02:45:18.711 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:18 +0000 (0:00:00.265) 0:02:14.801 ********* 2021-03-09T02:45:19.023 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-03-09T02:45:19.025 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-09T02:45:19.266 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-09T02:45:19.267 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-09T02:45:19.528 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-09T02:45:19.530 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-09T02:45:19.772 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-09T02:45:19.784 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-09T02:45:20.014 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-09T02:45:20.042 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:20 +0000 (0:00:01.331) 0:02:16.132 ********* 2021-03-09T02:45:20.052 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-03-09T02:45:20.081 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:20 +0000 (0:00:00.038) 0:02:16.171 ********* 2021-03-09T02:45:20.192 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-03-09T02:45:20.194 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-03-09T02:45:20.218 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:20 +0000 (0:00:00.136) 0:02:16.308 ********* 2021-03-09T02:45:20.235 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-03-09T02:45:20.262 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:20 +0000 (0:00:00.044) 0:02:16.352 ********* 2021-03-09T02:45:20.373 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-03-09T02:45:20.375 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-09T02:45:20.417 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-09T02:45:20.503 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-09T02:45:20.547 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-09T02:45:20.611 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-09T02:45:20.664 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-09T02:45:20.728 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-09T02:45:20.739 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:20.768 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:20 +0000 (0:00:00.505) 0:02:16.858 ********* 2021-03-09T02:45:20.786 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-03-09T02:45:20.787 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:20.815 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:20 +0000 (0:00:00.047) 0:02:16.906 ********* 2021-03-09T02:45:20.822 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-03-09T02:45:20.824 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:20.848 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:20 +0000 (0:00:00.032) 0:02:16.938 ********* 2021-03-09T02:45:20.855 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:20.881 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:20 +0000 (0:00:00.032) 0:02:16.971 ********* 2021-03-09T02:45:20.887 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-03-09T02:45:20.888 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:20.915 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:20 +0000 (0:00:00.034) 0:02:17.005 ********* 2021-03-09T02:45:20.922 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:20.958 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:20 +0000 (0:00:00.042) 0:02:17.048 ********* 2021-03-09T02:45:21.062 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-03-09T02:45:21.064 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-03-09T02:45:21.090 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:21 +0000 (0:00:00.132) 0:02:17.180 ********* 2021-03-09T02:45:21.110 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2021-03-09T02:45:21.140 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:21 +0000 (0:00:00.048) 0:02:17.229 ********* 2021-03-09T02:45:21.148 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-03-09T02:45:21.149 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-03-09T02:45:21.177 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:21 +0000 (0:00:00.037) 0:02:17.267 ********* 2021-03-09T02:45:21.184 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-03-09T02:45:21.210 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:21 +0000 (0:00:00.033) 0:02:17.301 ********* 2021-03-09T02:45:21.667 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-03-09T02:45:21.669 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true} changed: [smithi144.front.sepia.ceph.com] => {"changed": true} 2021-03-09T02:45:21.693 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:21 +0000 (0:00:00.482) 0:02:17.783 ********* 2021-03-09T02:45:21.714 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:21 +0000 (0:00:00.020) 0:02:17.804 ********* 2021-03-09T02:45:22.152 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-03-09T02:45:22.154 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-03-09T02:45:22.176 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.462) 0:02:18.266 ********* 2021-03-09T02:45:22.196 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.020) 0:02:18.286 ********* 2021-03-09T02:45:22.213 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-03-09T02:45:22.215 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:22.236 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.039) 0:02:18.326 ********* 2021-03-09T02:45:22.243 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:22.272 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.036) 0:02:18.362 ********* 2021-03-09T02:45:22.278 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-03-09T02:45:22.280 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:22.305 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.032) 0:02:18.395 ********* 2021-03-09T02:45:22.313 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:22.338 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.033) 0:02:18.428 ********* 2021-03-09T02:45:22.345 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-03-09T02:45:22.347 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:22.374 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.035) 0:02:18.464 ********* 2021-03-09T02:45:22.381 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:22.409 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.035) 0:02:18.499 ********* 2021-03-09T02:45:22.416 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-03-09T02:45:22.417 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:22.442 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.033) 0:02:18.532 ********* 2021-03-09T02:45:22.449 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:22.476 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.033) 0:02:18.566 ********* 2021-03-09T02:45:22.608 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-03-09T02:45:22.610 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-03-09T02:45:22.635 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.159) 0:02:18.725 ********* 2021-03-09T02:45:22.654 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2021-03-09T02:45:22.683 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.047) 0:02:18.773 ********* 2021-03-09T02:45:22.788 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-03-09T02:45:22.790 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-03-09T02:45:22.836 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.152) 0:02:18.925 ********* 2021-03-09T02:45:22.842 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2021-03-09T02:45:22.872 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:22 +0000 (0:00:00.037) 0:02:18.962 ********* 2021-03-09T02:45:23.228 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-03-09T02:45:23.230 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-03-09T02:45:23.266 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:23 +0000 (0:00:00.393) 0:02:19.356 ********* 2021-03-09T02:45:23.274 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1238, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} 2021-03-09T02:45:23.302 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:23 +0000 (0:00:00.036) 0:02:19.393 ********* 2021-03-09T02:45:23.622 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-03-09T02:45:23.624 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-03-09T02:45:23.651 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:23 +0000 (0:00:00.348) 0:02:19.741 ********* 2021-03-09T02:45:23.673 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2021-03-09T02:45:23.703 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:23 +0000 (0:00:00.051) 0:02:19.793 ********* 2021-03-09T02:45:24.572 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-03-09T02:45:24.573 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:45:24.599 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:24 +0000 (0:00:00.896) 0:02:20.689 ********* 2021-03-09T02:45:24.617 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:45:24.644 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:24 +0000 (0:00:00.044) 0:02:20.734 ********* 2021-03-09T02:45:24.797 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-03-09T02:45:24.799 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.044623", "end": "2021-03-09 02:45:24.317038", "rc": 0, "start": "2021-03-09 02:45:24.272415", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-03-09T02:45:24.822 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:24 +0000 (0:00:00.178) 0:02:20.913 ********* 2021-03-09T02:45:24.840 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.045233", "end": "2021-03-09 02:45:24.571952", "rc": 0, "start": "2021-03-09 02:45:24.526719", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2021-03-09T02:45:24.864 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:24 +0000 (0:00:00.041) 0:02:20.954 ********* 2021-03-09T02:45:24.871 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-03-09T02:45:24.873 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:24.897 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:24 +0000 (0:00:00.032) 0:02:20.987 ********* 2021-03-09T02:45:24.904 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:24.934 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:24 +0000 (0:00:00.037) 0:02:21.024 ********* 2021-03-09T02:45:24.942 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-03-09T02:45:24.944 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:24.972 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:24 +0000 (0:00:00.037) 0:02:21.062 ********* 2021-03-09T02:45:24.979 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:25.007 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:25 +0000 (0:00:00.035) 0:02:21.097 ********* 2021-03-09T02:45:25.865 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-03-09T02:45:25.867 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:45:25.894 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:25 +0000 (0:00:00.887) 0:02:21.985 ********* 2021-03-09T02:45:25.896 INFO:teuthology.task.ansible.out: 2021-03-09T02:45:25.904 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:45:25.906 INFO:teuthology.task.ansible.out: 2021-03-09T02:45:25.931 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:25 +0000 (0:00:00.036) 0:02:22.021 ********* 2021-03-09T02:45:25.933 INFO:teuthology.task.ansible.out: 2021-03-09T02:45:25.940 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-03-09T02:45:25.942 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:25.972 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:25 +0000 (0:00:00.040) 0:02:22.062 ********* 2021-03-09T02:45:25.980 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:26.008 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.035) 0:02:22.098 ********* 2021-03-09T02:45:26.314 INFO:teuthology.task.ansible.out: 2021-03-09T02:45:26.316 INFO:teuthology.task.ansible.out:TASK [testnode : Create the ntp.conf file.] ************************************ ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-03-09T02:45:26.349 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.341) 0:02:22.439 ********* 2021-03-09T02:45:26.357 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2021-03-09T02:45:26.384 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.035) 0:02:22.475 ********* 2021-03-09T02:45:26.391 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-03-09T02:45:26.393 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:26.424 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.039) 0:02:22.514 ********* 2021-03-09T02:45:26.433 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:26.464 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.040) 0:02:22.555 ********* 2021-03-09T02:45:26.642 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-03-09T02:45:26.643 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Tue 2021-03-09 02:42:07 UTC", "ActiveEnterTimestampMonotonic": "16265870", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target system.slice network.target systemd-journald.socket -.mount systemd-tmpfiles-setup.service sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2021-03-09 02:42:07 UTC", "AssertTimestampMonotonic": "15984169", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2021-03-09 02:42:07 UTC", "ConditionTimestampMonotonic": "15984169", "ConfigurationDirectoryMode": "0755", "Conflicts": "systemd-timesyncd.service shutdown.target", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-03-09T02:45:26.645 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1260", "ExecMainStartTimestamp": "Tue 2021-03-09 02:42:07 UTC", "ExecMainStartTimestampMonotonic": "16265859", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2021-03-09 02:42:07 UTC", "InactiveExitTimestampMonotonic": "15984938", "InvocationID": "076a740ce24641128d31ac18b6b5083f", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-03-09T02:45:26.647 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1260", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice sysinit.target -.mount", "RequiresMountsFor": "/var/tmp /tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2021-03-09 02:42:07 UTC", "S 2021-03-09T02:45:26.649 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "16265870", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2021-03-09 02:42:07 UTC", "WatchdogTimestampMonotonic": "16265869", "WatchdogUSec": "0"}} 2021-03-09T02:45:26.683 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.218) 0:02:22.773 ********* 2021-03-09T02:45:26.693 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "ntp", "state": "started", "status": {"ActiveEnterTimestamp": "Tue 2021-03-09 02:42:13 UTC", "ActiveEnterTimestampMonotonic": "17077405", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "network.target system.slice sysinit.target systemd-journald.socket systemd-tmpfiles-setup.service basic.target -.mount", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "yes", "AssertTimestamp": "Tue 2021-03-09 02:42:13 UTC", "AssertTimestampMonotonic": "16940071", "Before": "shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "yes", "ConditionTimestamp": "Tue 2021-03-09 02:42:13 UTC", "ConditionTimestampMonotonic": "16940070", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target systemd-timesyncd.service", "ControlGroup": "/system.slice/ntp.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "Documen 2021-03-09T02:45:26.694 INFO:teuthology.task.ansible.out:tation": "man:ntpd(8)", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1084", "ExecMainStartTimestamp": "Tue 2021-03-09 02:42:13 UTC", "ExecMainStartTimestampMonotonic": "17077390", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/lib/ntp/ntp-systemd-wrapper ; argv[]=/usr/lib/ntp/ntp-systemd-wrapper ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/ntp.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "ntp.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Tue 2021-03-09 02:42:13 UTC", "InactiveExitTimestampMonotonic": "16940727", "InvocationID": "ef036e907d9540ef8f25d3acdfe76c19", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "0", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "4096", "LimitNOFILESoft": "1024", "LimitNPROC": "128029", "LimitNPROCSoft": "128029", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "Limi 2021-03-09T02:45:26.696 INFO:teuthology.task.ansible.out:tRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "128029", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "1084", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "ntp.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice -.mount sysinit.target", "RequiresMountsFor": "/tmp /var/tmp", "Restart": "no", "RestartUSec": "100ms", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitIntervalUSec": "10s", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2021-03-09 02:42:13 UTC", "S 2021-03-09T02:45:26.698 INFO:teuthology.task.ansible.out:tateChangeTimestampMonotonic": "17077405", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SuccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "2", "TasksMax": "4915", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "forking", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "enabled", "UtmpMode": "init", "WantedBy": "multi-user.target", "WatchdogTimestamp": "Tue 2021-03-09 02:42:13 UTC", "WatchdogTimestampMonotonic": "17077405", "WatchdogUSec": "0"}} 2021-03-09T02:45:26.719 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.035) 0:02:22.809 ********* 2021-03-09T02:45:26.726 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-09T02:45:26.727 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-03-09T02:45:26.750 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.030) 0:02:22.840 ********* 2021-03-09T02:45:26.758 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-03-09T02:45:26.783 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.032) 0:02:22.873 ********* 2021-03-09T02:45:26.788 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-09T02:45:26.790 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:26.816 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.033) 0:02:22.906 ********* 2021-03-09T02:45:26.823 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:26.848 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.031) 0:02:22.938 ********* 2021-03-09T02:45:26.855 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-03-09T02:45:26.857 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:26.881 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.033) 0:02:22.971 ********* 2021-03-09T02:45:26.888 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:26.913 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:26 +0000 (0:00:00.032) 0:02:23.003 ********* 2021-03-09T02:45:27.756 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-03-09T02:45:27.758 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:45:27.781 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:27 +0000 (0:00:00.867) 0:02:23.871 ********* 2021-03-09T02:45:27.798 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:45:27.822 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:27 +0000 (0:00:00.040) 0:02:23.912 ********* 2021-03-09T02:45:27.828 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-03-09T02:45:27.830 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:27.856 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:27 +0000 (0:00:00.033) 0:02:23.946 ********* 2021-03-09T02:45:27.863 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:27.890 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:27 +0000 (0:00:00.034) 0:02:23.980 ********* 2021-03-09T02:45:27.992 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-03-09T02:45:27.994 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-03-09T02:45:28.020 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:28 +0000 (0:00:00.130) 0:02:24.110 ********* 2021-03-09T02:45:28.027 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2021-03-09T02:45:28.055 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:28 +0000 (0:00:00.035) 0:02:24.145 ********* 2021-03-09T02:45:28.404 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-03-09T02:45:28.405 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-03-09T02:45:28.436 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:28 +0000 (0:00:00.380) 0:02:24.526 ********* 2021-03-09T02:45:28.467 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1238, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2021-03-09T02:45:28.492 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:28 +0000 (0:00:00.056) 0:02:24.582 ********* 2021-03-09T02:45:29.478 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-03-09T02:45:29.480 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-03-09T02:45:29.504 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:29 +0000 (0:00:01.012) 0:02:25.595 ********* 2021-03-09T02:45:29.523 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-03-09T02:45:29.552 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:29 +0000 (0:00:00.047) 0:02:25.643 ********* 2021-03-09T02:45:30.044 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-03-09T02:45:30.046 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false} 2021-03-09T02:45:30.082 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.529) 0:02:26.172 ********* 2021-03-09T02:45:30.102 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false} 2021-03-09T02:45:30.127 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.045) 0:02:26.217 ********* 2021-03-09T02:45:30.134 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-03-09T02:45:30.136 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-03-09T02:45:30.165 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.037) 0:02:26.255 ********* 2021-03-09T02:45:30.172 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-03-09T02:45:30.201 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.036) 0:02:26.291 ********* 2021-03-09T02:45:30.303 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-03-09T02:45:30.305 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1615257749.0317204, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1615257749.0277205, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 262873, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1615257749.0277205, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3476867264", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-03-09T02:45:30.333 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.132) 0:02:26.423 ********* 2021-03-09T02:45:30.356 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1615257754.8834722, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1615257754.879472, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 262873, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1615257754.879472, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "3476867264", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2021-03-09T02:45:30.384 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.050) 0:02:26.474 ********* 2021-03-09T02:45:30.727 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-03-09T02:45:30.729 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "msg": ""} ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2021-03-09T02:45:30.754 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.369) 0:02:26.844 ********* 2021-03-09T02:45:30.777 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.022) 0:02:26.867 ********* 2021-03-09T02:45:30.784 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-03-09T02:45:30.786 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-03-09T02:45:30.810 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.033) 0:02:26.901 ********* 2021-03-09T02:45:30.817 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-03-09T02:45:30.841 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.030) 0:02:26.931 ********* 2021-03-09T02:45:30.847 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-03-09T02:45:30.849 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:30.872 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.030) 0:02:26.962 ********* 2021-03-09T02:45:30.879 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:30.904 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:30 +0000 (0:00:00.031) 0:02:26.994 ********* 2021-03-09T02:45:31.004 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-03-09T02:45:31.006 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-03-09T02:45:31.029 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.124) 0:02:27.119 ********* 2021-03-09T02:45:31.047 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-03-09T02:45:31.072 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.042) 0:02:27.162 ********* 2021-03-09T02:45:31.172 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-03-09T02:45:31.174 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-03-09T02:45:31.197 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.124) 0:02:27.287 ********* 2021-03-09T02:45:31.214 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-03-09T02:45:31.236 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.039) 0:02:27.327 ********* 2021-03-09T02:45:31.242 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-03-09T02:45:31.244 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.267 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.030) 0:02:27.357 ********* 2021-03-09T02:45:31.273 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.298 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.031) 0:02:27.388 ********* 2021-03-09T02:45:31.306 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-03-09T02:45:31.308 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.318 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.319 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.349 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.050) 0:02:27.439 ********* 2021-03-09T02:45:31.357 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.359 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.361 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.387 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.038) 0:02:27.477 ********* 2021-03-09T02:45:31.395 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-03-09T02:45:31.397 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.408 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.410 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi092.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.437 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.049) 0:02:27.527 ********* 2021-03-09T02:45:31.446 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.448 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi144.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.460 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.461 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.488 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.051) 0:02:27.579 ********* 2021-03-09T02:45:31.509 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.020) 0:02:27.599 ********* 2021-03-09T02:45:31.516 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-03-09T02:45:31.517 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.542 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.033) 0:02:27.633 ********* 2021-03-09T02:45:31.549 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-03-09T02:45:31.551 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.576 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.033) 0:02:27.666 ********* 2021-03-09T02:45:31.582 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-03-09T02:45:31.584 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.610 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.033) 0:02:27.700 ********* 2021-03-09T02:45:31.616 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-03-09T02:45:31.618 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.644 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.034) 0:02:27.734 ********* 2021-03-09T02:45:31.650 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-03-09T02:45:31.652 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.675 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.031) 0:02:27.765 ********* 2021-03-09T02:45:31.682 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-03-09T02:45:31.683 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.708 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.032) 0:02:27.798 ********* 2021-03-09T02:45:31.714 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-03-09T02:45:31.716 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.743 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.034) 0:02:27.833 ********* 2021-03-09T02:45:31.750 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-03-09T02:45:31.752 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.778 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.035) 0:02:27.868 ********* 2021-03-09T02:45:31.785 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-03-09T02:45:31.786 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.812 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.033) 0:02:27.902 ********* 2021-03-09T02:45:31.818 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-03-09T02:45:31.820 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.845 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.033) 0:02:27.936 ********* 2021-03-09T02:45:31.852 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-03-09T02:45:31.853 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.879 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.033) 0:02:27.969 ********* 2021-03-09T02:45:31.885 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-03-09T02:45:31.887 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.912 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.033) 0:02:28.003 ********* 2021-03-09T02:45:31.919 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-03-09T02:45:31.921 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.947 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.034) 0:02:28.037 ********* 2021-03-09T02:45:31.954 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-03-09T02:45:31.955 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:31.983 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:31 +0000 (0:00:00.035) 0:02:28.073 ********* 2021-03-09T02:45:32.010 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-03-09T02:45:32.011 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:32.038 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:32 +0000 (0:00:00.054) 0:02:28.128 ********* 2021-03-09T02:45:32.098 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-03-09T02:45:32.101 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.011124", "end": "2021-03-09 02:45:31.616600", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-03-09 02:45:31.605476", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:32.194 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.008817", "end": "2021-03-09 02:45:31.926483", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-03-09 02:45:31.917666", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:32.226 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006501", "end": "2021-03-09 02:45:31.749191", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-03-09 02:45:31.742690", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:32.320 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.006034", "end": "2021-03-09 02:45:32.051296", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-03-09 02:45:32.045262", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:32.342 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.005458", "end": "2021-03-09 02:45:31.863935", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-03-09 02:45:31.858477", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:32.426 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.004601", "end": "2021-03-09 02:45:32.163541", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-03-09 02:45:32.158940", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:32.458 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.007315", "end": "2021-03-09 02:45:31.978548", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-03-09 02:45:31.971233", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:32.483 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:32 +0000 (0:00:00.445) 0:02:28.573 ********* 2021-03-09T02:45:32.510 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-03-09T02:45:32.533 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:32 +0000 (0:00:00.050) 0:02:28.623 ********* 2021-03-09T02:45:32.551 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-03-09T02:45:32.551 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.007574", "end": "2021-03-09 02:45:32.283402", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-03-09 02:45:32.275828", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:32.578 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:32 +0000 (0:00:00.044) 0:02:28.668 ********* 2021-03-09T02:45:32.605 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-03-09T02:45:32.635 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:32 +0000 (0:00:00.057) 0:02:28.725 ********* 2021-03-09T02:45:32.672 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-03-09T02:45:32.674 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-03-09T02:45:32.772 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2021-03-09T02:45:32.855 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-03-09T02:45:32.883 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:32 +0000 (0:00:00.247) 0:02:28.973 ********* 2021-03-09T02:45:32.904 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:32 +0000 (0:00:00.021) 0:02:28.994 ********* 2021-03-09T02:45:33.630 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi092.front.sepia.ceph.com] 2021-03-09T02:45:33.665 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] 2021-03-09T02:45:33.694 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:33 +0000 (0:00:00.789) 0:02:29.784 ********* 2021-03-09T02:45:33.737 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:33 +0000 (0:00:00.043) 0:02:29.827 ********* 2021-03-09T02:45:33.744 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-09T02:45:33.746 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-03-09T02:45:33.771 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:33 +0000 (0:00:00.033) 0:02:29.861 ********* 2021-03-09T02:45:33.778 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "apt"}, "changed": false} 2021-03-09T02:45:33.803 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:33 +0000 (0:00:00.032) 0:02:29.893 ********* 2021-03-09T02:45:33.810 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-09T02:45:33.812 INFO:teuthology.task.ansible.out:skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:33.842 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:33 +0000 (0:00:00.038) 0:02:29.932 ********* 2021-03-09T02:45:33.853 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:33.882 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:33 +0000 (0:00:00.039) 0:02:29.972 ********* 2021-03-09T02:45:33.901 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-03-09T02:45:33.903 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-03-09T02:45:33.932 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:33 +0000 (0:00:00.049) 0:02:30.022 ********* 2021-03-09T02:45:33.940 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml) => {"ansible_facts": {"container_packages": ["docker.io"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/apt_systems.yml"} 2021-03-09T02:45:33.979 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:33 +0000 (0:00:00.046) 0:02:30.069 ********* 2021-03-09T02:45:34.871 INFO:teuthology.task.ansible.out: 2021-03-09T02:45:34.872 INFO:teuthology.task.ansible.out:TASK [container-host : Install container packages] ***************************** ok: [smithi092.front.sepia.ceph.com] => {"cache_update_time": 1615257771, "cache_updated": false, "changed": false} 2021-03-09T02:45:34.895 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:34 +0000 (0:00:00.916) 0:02:30.985 ********* 2021-03-09T02:45:34.912 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"cache_update_time": 1615257776, "cache_updated": false, "changed": false} 2021-03-09T02:45:34.938 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:34 +0000 (0:00:00.042) 0:02:31.028 ********* 2021-03-09T02:45:34.945 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** skipping: [smithi092.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:34.971 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:34 +0000 (0:00:00.032) 0:02:31.061 ********* 2021-03-09T02:45:34.978 INFO:teuthology.task.ansible.out:skipping: [smithi144.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-03-09T02:45:35.004 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:35 +0000 (0:00:00.033) 0:02:31.094 ********* 2021-03-09T02:45:35.010 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-03-09T02:45:35.012 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-03-09T02:45:35.037 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:35 +0000 (0:00:00.032) 0:02:31.127 ********* 2021-03-09T02:45:35.043 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/docker/daemon.json"}, "changed": false} 2021-03-09T02:45:35.068 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:35 +0000 (0:00:00.031) 0:02:31.159 ********* 2021-03-09T02:45:35.190 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-03-09T02:45:35.191 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-09T02:45:35.223 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-09T02:45:35.298 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-09T02:45:35.325 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:35 +0000 (0:00:00.256) 0:02:31.415 ********* 2021-03-09T02:45:35.343 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2021-03-09T02:45:35.374 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:35 +0000 (0:00:00.048) 0:02:31.464 ********* 2021-03-09T02:45:35.697 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-03-09T02:45:35.699 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615257935.3921757-21177-258329869255938/source", "state": "file", "uid": 0} 2021-03-09T02:45:35.729 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615257935.4344637-21187-192300271384754/source", "state": "file", "uid": 0} 2021-03-09T02:45:36.022 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615257935.7235913-21177-50622864639730/source", "state": "file", "uid": 0} 2021-03-09T02:45:36.053 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:36 +0000 (0:00:00.678) 0:02:32.143 ********* 2021-03-09T02:45:36.072 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": true, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "md5sum": "a3fa0991425692932fcf703c1a9a483e", "mode": "0644", "owner": "root", "size": 2334, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615257935.7565675-21187-2758389819928/source", "state": "file", "uid": 0} 2021-03-09T02:45:36.101 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:36 +0000 (0:00:00.047) 0:02:32.191 ********* 2021-03-09T02:45:39.538 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-03-09T02:45:39.540 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-eJGIJs-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-eJGIJs-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-03-09T02:45:39.542 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-03-09T02:45:39.563 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:39 +0000 (0:00:03.461) 0:02:35.653 ********* 2021-03-09T02:45:39.623 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["/usr/bin/pip2", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "", "stderr_lines": [], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-MlNjXR-build\nCollecting docopt (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\nCollecting toml (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl\nCollecting typing (from registries-conf-ctl==1.0)\n Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl\nBuilding wheels for collected packages: docopt\n Running setup.py bdist_wheel for docopt: started\n Running setup.py bdist_wheel for docopt: finished with status 'done'\n Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\nSuccessfully built docopt\nInstalling collected packages: docopt, toml, typing, registries-conf-ctl\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-MlNjXR-build", "Collecting docopt (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packag 2021-03-09T02:45:39.625 INFO:teuthology.task.ansible.out:es/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz", "Collecting toml (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/44/6f/7120676b6d73228c96e17f1f794d8ab046fc910d781c8d151120c3f1569e/toml-0.10.2-py2.py3-none-any.whl", "Collecting typing (from registries-conf-ctl==1.0)", " Downloading https://files.pythonhosted.org/packages/3b/c0/e44213fcb799eac02881e2485724ba5b0914600bc9df6ed922e364fdc059/typing-3.7.4.3-py2-none-any.whl", "Building wheels for collected packages: docopt", " Running setup.py bdist_wheel for docopt: started", " Running setup.py bdist_wheel for docopt: finished with status 'done'", " Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e", "Successfully built docopt", "Installing collected packages: docopt, toml, typing, registries-conf-ctl", " Running setup.py install for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed docopt-0.6.2 registries-conf-ctl-1.0 toml-0.10.2 typing-3.7.4.3"], "version": null, "virtualenv": null} 2021-03-09T02:45:39.650 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:39 +0000 (0:00:00.087) 0:02:35.740 ********* 2021-03-09T02:45:39.711 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-03-09T02:45:39.713 INFO:teuthology.task.ansible.out:ok: [smithi144.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-03-09T02:45:39.740 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:39 +0000 (0:00:00.090) 0:02:35.830 ********* 2021-03-09T02:45:39.790 INFO:teuthology.task.ansible.out:ok: [smithi092.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-03-09T02:45:39.819 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:39 +0000 (0:00:00.078) 0:02:35.909 ********* 2021-03-09T02:45:40.146 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/docker/daemon.json if necessary] ************ 2021-03-09T02:45:40.148 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615257939.8131008-21781-99842433746454/source", "state": "file", "uid": 0} 2021-03-09T02:45:40.171 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:40 +0000 (0:00:00.352) 0:02:36.262 ********* 2021-03-09T02:45:40.180 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "checksum": "bf21a9e8fbc5a3846fb05b4fa0859e0917b2202f", "dest": "/etc/docker/daemon.json", "gid": 0, "group": "root", "md5sum": "99914b932bd37a50b983c5e7c90ae93b", "mode": "0644", "owner": "root", "size": 2, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1615257939.8743072-21791-59627814787849/source", "state": "file", "uid": 0} 2021-03-09T02:45:40.209 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:40 +0000 (0:00:00.037) 0:02:36.299 ********* 2021-03-09T02:45:40.436 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-03-09T02:45:40.438 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.128558", "end": "2021-03-09 02:45:40.174888", "rc": 0, "start": "2021-03-09 02:45:40.046330", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:40.462 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:40 +0000 (0:00:00.252) 0:02:36.552 ********* 2021-03-09T02:45:40.471 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.120552", "end": "2021-03-09 02:45:39.994723", "rc": 0, "start": "2021-03-09 02:45:39.874171", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-03-09T02:45:40.499 INFO:teuthology.task.ansible.out:Tuesday 09 March 2021 02:45:40 +0000 (0:00:00.037) 0:02:36.589 ********* 2021-03-09T02:45:43.745 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-03-09T02:45:43.747 INFO:teuthology.task.ansible.out:changed: [smithi144.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "firewalld.service network-online.target system.slice systemd-journald.socket docker.socket containerd.service basic.target sysinit.target", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMa 2021-03-09T02:45:43.748 INFO:teuthology.task.ansible.out:inStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "12802 2021-03-09T02:45:43.750 INFO:teuthology.task.ansible.out:9", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "docker.socket system.slice sysinit.target", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2021-03-09 02:42:37 UTC", "StateChangeTimestampMonotonic": "41001069", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "Su 2021-03-09T02:45:43.752 INFO:teuthology.task.ansible.out:ccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "containerd.service network-online.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-09T02:45:43.900 INFO:teuthology.task.ansible.out:changed: [smithi092.front.sepia.ceph.com] => {"changed": true, "name": "docker", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target containerd.service sysinit.target systemd-journald.socket system.slice network-online.target docker.socket firewalld.service", "AllowIsolate": "no", "AmbientCapabilities": "", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "CacheDirectoryMode": "0755", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "cap_chown cap_dac_override cap_dac_read_search cap_fowner cap_fsetid cap_kill cap_setgid cap_setuid cap_setpcap cap_linux_immutable cap_net_bind_service cap_net_broadcast cap_net_admin cap_net_raw cap_ipc_lock cap_ipc_owner cap_sys_module cap_sys_rawio cap_sys_chroot cap_sys_ptrace cap_sys_pacct cap_sys_admin cap_sys_boot cap_sys_nice cap_sys_resource cap_sys_time cap_sys_tty_config cap_mknod cap_lease cap_audit_write cap_audit_control cap_setfcap cap_mac_override cap_mac_admin cap_syslog cap_wake_alarm cap_block_suspend", "CollectMode": "inactive", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConfigurationDirectoryMode": "0755", "Conflicts": "shutdown.target", "ConsistsOf": "docker.socket", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "yes", "DelegateControllers": "cpu cpuacct io blkio memory devices pids", "Description": "Docker Application Container Engine", "DevicePolicy": "auto", "Documentation": "https://docs.docker.com", "DynamicUser": "no", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMa 2021-03-09T02:45:43.902 INFO:teuthology.task.ansible.out:inStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -s HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/bin/dockerd ; argv[]=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/lib/systemd/system/docker.service", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "Id": "docker.service", "IgnoreOnIsolate": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobRunningTimeoutUSec": "infinity", "JobTimeoutAction": "none", "JobTimeoutUSec": "infinity", "KeyringMode": "private", "KillMode": "process", "KillSignal": "15", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "67108864", "LimitMEMLOCKSoft": "67108864", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "infinity", "LimitNOFILESoft": "infinity", "LimitNPROC": "infinity", "LimitNPROCSoft": "infinity", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "128029", "LimitSIGPENDINGSoft": "12802 2021-03-09T02:45:43.903 INFO:teuthology.task.ansible.out:9", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LoadState": "loaded", "LockPersonality": "no", "LogLevelMax": "-1", "LogsDirectoryMode": "0755", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "[not set]", "MemoryDenyWriteExecute": "no", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemorySwapMax": "infinity", "MountAPIVFS": "no", "MountFlags": "", "NFileDescriptorStore": "0", "NRestarts": "0", "Names": "docker.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "main", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "Perpetual": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "PrivateUsers": "no", "ProtectControlGroups": "no", "ProtectHome": "no", "ProtectKernelModules": "no", "ProtectKernelTunables": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "RemoveIPC": "no", "Requires": "system.slice docker.socket sysinit.target", "Restart": "always", "RestartUSec": "2s", "RestrictNamespaces": "no", "RestrictRealtime": "no", "RestrictSUIDSGID": "no", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "RuntimeDirectoryPreserve": "no", "RuntimeMaxUSec": "infinity", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "3", "StartLimitIntervalUSec": "1min", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StateChangeTimestamp": "Tue 2021-03-09 02:42:31 UTC", "StateChangeTimestampMonotonic": "40006628", "StateDirectoryMode": "0755", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "Su 2021-03-09T02:45:43.905 INFO:teuthology.task.ansible.out:ccessAction": "none", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "infinity", "TimeoutStartUSec": "infinity", "TimeoutStopUSec": "infinity", "TimerSlackNSec": "50000", "Transient": "no", "TriggeredBy": "docker.socket", "Type": "notify", "UID": "[not set]", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "UtmpMode": "init", "Wants": "network-online.target containerd.service", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-03-09T02:45:43.973 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-03-09T02:45:43.976 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2021-03-09T02:45:43.976 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2021-03-09T02:45:43.977 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* smithi092.front.sepia.ceph.com : ok=127 changed=35 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 smithi144.front.sepia.ceph.com : ok=127 changed=35 unreachable=0 failed=0 skipped=184 rescued=0 ignored=0 Tuesday 09 March 2021 02:45:43 +0000 (0:00:03.477) 0:02:40.067 ********* =============================================================================== 2021-03-09T02:45:43.979 INFO:teuthology.task.ansible.out:Create all admin users with sudo access. ------------------------------- 20.96s users : Update authorized_keys using the keys repo --------------------- 18.13s testnode : Ensure packages are not present. ---------------------------- 10.04s testnode : Install packages --------------------------------------------- 8.74s testnode : Zap all non-root disks --------------------------------------- 8.44s Remove revoked users ---------------------------------------------------- 6.83s container-host : Restart docker service --------------------------------- 3.48s container-host : Install registries-conf-ctl ---------------------------- 3.46s common : Update apt cache ----------------------------------------------- 2.96s testnode : Update apt cache. -------------------------------------------- 2.96s testnode : Install apt keys --------------------------------------------- 2.13s Gathering Facts --------------------------------------------------------- 1.66s testnode : Blow away lingering OSD data and FSIDs ----------------------- 1.55s users : Update authorized_keys for each user with literal keys ---------- 1.43s testnode : Create logical volume(s) ------------------------------------- 1.33s common : Install Kerberos Packages (Debian) ----------------------------- 1.02s testnode : Install packages via pip ------------------------------------- 1.01s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.01s testnode : Make sure apt dependencies are installed --------------------- 1.01s testnode : Install python-apt ------------------------------------------- 0.97s 2021-03-09T02:45:44.348 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi092.front.sepia.ceph.com'), Remote(name='ubuntu@smithi144.front.sepia.ceph.com')] 2021-03-09T02:45:44.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-09T02:45:44.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi092.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-09T02:45:44.429 DEBUG:teuthology.orchestra.run.smithi092:> true 2021-03-09T02:45:44.619 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2021-03-09T02:45:44.620 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-03-09T02:45:44.695 DEBUG:teuthology.orchestra.run.smithi144:> true 2021-03-09T02:45:44.888 INFO:teuthology.run_tasks:Running task clock... 2021-03-09T02:45:44.905 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-03-09T02:45:44.905 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-03-09T02:45:44.908 DEBUG:teuthology.orchestra.run.smithi092:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-03-09T02:45:44.965 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-03-09T02:45:44.966 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: Command line: ntpd -gq 2021-03-09T02:45:44.967 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: proto: precision = 0.123 usec (-23) 2021-03-09T02:45:44.967 INFO:teuthology.orchestra.run.smithi092.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-09T02:45:44.968 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-09T02:45:44.968 INFO:teuthology.orchestra.run.smithi092.stderr:restrict ::: KOD does nothing without LIMITED. 2021-03-09T02:45:44.969 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: restrict ::: KOD does nothing without LIMITED. 2021-03-09T02:45:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: Listen and drop on 0 v6wildcard [::]:123 2021-03-09T02:45:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-03-09T02:45:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: Listen normally on 2 lo 127.0.0.1:123 2021-03-09T02:45:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: Listen normally on 3 enp3s0f1 172.21.15.92:123 2021-03-09T02:45:44.970 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: Listen normally on 4 lo [::1]:123 2021-03-09T02:45:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cd45%5]:123 2021-03-09T02:45:44.971 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:44 ntpd[9383]: Listening on routing socket on fd #22 for interface updates 2021-03-09T02:45:45.970 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:45 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:45.971 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:45 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:45.971 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:45 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:45.971 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:45 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:45.971 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:45 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:45.972 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:45 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:45.972 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:45 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:45.972 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:45 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:46.966 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:46 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:46.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:46 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:46.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:46 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:46.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:46 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:47.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:47 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:47.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:47 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:47.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:47 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:47.968 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:47 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:47.968 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:47 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:47.968 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:47 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:47.968 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:47 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:47.968 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:47 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:48.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:48 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:48.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:48 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:48.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:48 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:48.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:48 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:48.968 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:48 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:48.968 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:48 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:48.968 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:48 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:48.968 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:48 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:49.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:49 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:49.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:49 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:49.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:49 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:49.969 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:49 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:49.969 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:49 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:49.969 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:49 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:49.970 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:49 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:49.971 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:49 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:50.973 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:50 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:50.974 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:50 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:50.974 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:50 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:50.974 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:50 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:50.974 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:50 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:50.975 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:50 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:50.975 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:50 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:50.975 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:50 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:51.966 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:51 ntpd[9383]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:51.966 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:51 ntpd[9383]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:51.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:51 ntpd[9383]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:51.967 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:51 ntpd[9383]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:51.967 INFO:teuthology.orchestra.run.smithi092.stdout: 9 Mar 02:45:51 ntpd[9383]: ntpd: time set +0.452091 s 2021-03-09T02:45:51.968 INFO:teuthology.orchestra.run.smithi092.stdout:ntpd: time set +0.452091s 2021-03-09T02:45:51.968 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:51 ntpd[9383]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-03-09T02:45:51.968 INFO:teuthology.orchestra.run.smithi092.stderr: 9 Mar 02:45:51 ntpd[9383]: can't open /var/log/ntpstats/loopstats.20210309: Permission denied 2021-03-09T02:45:52.309 INFO:teuthology.orchestra.run.smithi092.stdout: remote refid st t when poll reach delay offset jitter 2021-03-09T02:45:52.310 INFO:teuthology.orchestra.run.smithi092.stdout:============================================================================== 2021-03-09T02:45:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-09T02:45:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-09T02:45:52.310 INFO:teuthology.orchestra.run.smithi092.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-09T02:45:52.311 INFO:teuthology.orchestra.run.smithi092.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-09T02:45:52.312 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-03-09T02:45:52.312 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-03-09T02:45:52.377 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: ntpd 4.2.8p10@1.3728-o (1): Starting 2021-03-09T02:45:52.377 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: Command line: ntpd -gq 2021-03-09T02:45:52.378 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: proto: precision = 0.120 usec (-23) 2021-03-09T02:45:52.378 INFO:teuthology.orchestra.run.smithi144.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-09T02:45:52.379 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2021-03-09T02:45:52.379 INFO:teuthology.orchestra.run.smithi144.stderr:restrict ::: KOD does nothing without LIMITED. 2021-03-09T02:45:52.386 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: restrict ::: KOD does nothing without LIMITED. 2021-03-09T02:45:52.387 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: Listen and drop on 0 v6wildcard [::]:123 2021-03-09T02:45:52.387 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2021-03-09T02:45:52.387 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: Listen normally on 2 lo 127.0.0.1:123 2021-03-09T02:45:52.387 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: Listen normally on 3 enp3s0f1 172.21.15.144:123 2021-03-09T02:45:52.388 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: Listen normally on 4 lo [::1]:123 2021-03-09T02:45:52.390 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7f5f%5]:123 2021-03-09T02:45:52.391 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:52 ntpd[9280]: Listening on routing socket on fd #22 for interface updates 2021-03-09T02:45:53.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:53 ntpd[9280]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:53.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:53 ntpd[9280]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:53.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:53 ntpd[9280]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:53.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:53 ntpd[9280]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:54.377 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:54 ntpd[9280]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:54.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:54 ntpd[9280]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:54.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:54 ntpd[9280]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:54.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:54 ntpd[9280]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:55.377 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:55 ntpd[9280]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:55.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:55 ntpd[9280]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:55.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:55 ntpd[9280]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:55.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:55 ntpd[9280]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:55.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:55 ntpd[9280]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:55.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:55 ntpd[9280]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:55.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:55 ntpd[9280]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:55.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:55 ntpd[9280]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:56.377 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:56 ntpd[9280]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:56.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:56 ntpd[9280]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:56.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:56 ntpd[9280]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:56.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:56 ntpd[9280]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:57.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:57.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:57.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:57.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:57.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:57.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:57.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:57.380 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:57.380 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:57.380 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:57.380 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:57.381 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:57 ntpd[9280]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:58.377 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:58 ntpd[9280]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:58.377 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:58 ntpd[9280]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:58.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:58 ntpd[9280]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:58.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:58 ntpd[9280]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:59.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:59 ntpd[9280]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:59.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:59 ntpd[9280]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:59.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:59 ntpd[9280]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:59.378 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:59 ntpd[9280]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:59.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:59 ntpd[9280]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2021-03-09T02:45:59.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:59 ntpd[9280]: can't open /var/log/ntpstats/rawstats.20210309: Permission denied 2021-03-09T02:45:59.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:59 ntpd[9280]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2021-03-09T02:45:59.379 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:59 ntpd[9280]: can't open /var/log/ntpstats/peerstats.20210309: Permission denied 2021-03-09T02:45:59.380 INFO:teuthology.orchestra.run.smithi144.stdout: 9 Mar 02:45:59 ntpd[9280]: ntpd: time set +0.241604 s 2021-03-09T02:45:59.380 INFO:teuthology.orchestra.run.smithi144.stdout:ntpd: time set +0.241604s 2021-03-09T02:45:59.381 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:59 ntpd[9280]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2021-03-09T02:45:59.381 INFO:teuthology.orchestra.run.smithi144.stderr: 9 Mar 02:45:59 ntpd[9280]: can't open /var/log/ntpstats/loopstats.20210309: Permission denied 2021-03-09T02:45:59.442 INFO:teuthology.orchestra.run.smithi144.stdout: remote refid st t when poll reach delay offset jitter 2021-03-09T02:45:59.442 INFO:teuthology.orchestra.run.smithi144.stdout:============================================================================== 2021-03-09T02:45:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-09T02:45:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-09T02:45:59.443 INFO:teuthology.orchestra.run.smithi144.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-09T02:45:59.444 INFO:teuthology.orchestra.run.smithi144.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2021-03-09T02:45:59.444 INFO:teuthology.run_tasks:Running task install... 2021-03-09T02:45:59.457 DEBUG:teuthology.task.install:project ceph 2021-03-09T02:45:59.458 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '9e448db9e385343da144613d840349182443aeee'}} 2021-03-09T02:45:59.458 DEBUG:teuthology.task.install:config {'extra_packages': ['rbd-mirror'], 'sha1': '9e448db9e385343da144613d840349182443aeee'} 2021-03-09T02:45:59.459 INFO:teuthology.task.install:Using flavor: basic 2021-03-09T02:45:59.477 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-03-09T02:45:59.477 INFO:teuthology.task.install:extra packages: ['rbd-mirror'] 2021-03-09T02:45:59.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo lsb_release -is 2021-03-09T02:45:59.530 INFO:teuthology.orchestra.run.smithi092.stdout:Ubuntu 2021-03-09T02:45:59.535 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-09T02:45:59.535 DEBUG:teuthology.orchestra.run.smithi144:> sudo lsb_release -is 2021-03-09T02:45:59.536 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-key list | grep Ceph 2021-03-09T02:45:59.591 INFO:teuthology.orchestra.run.smithi144.stdout:Ubuntu 2021-03-09T02:45:59.594 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-09T02:45:59.595 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-key list | grep Ceph 2021-03-09T02:45:59.670 INFO:teuthology.orchestra.run.smithi092.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-03-09T02:45:59.691 INFO:teuthology.orchestra.run.smithi144.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2021-03-09T02:45:59.730 INFO:teuthology.orchestra.run.smithi092.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-03-09T02:45:59.730 INFO:teuthology.orchestra.run.smithi092.stdout:uid [ unknown] Ceph.com (release key) 2021-03-09T02:45:59.731 DEBUG:teuthology.orchestra.run.smithi092:> sudo lsb_release -is 2021-03-09T02:45:59.745 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2021-03-09T02:45:59.746 INFO:teuthology.orchestra.run.smithi144.stdout:uid [ unknown] Ceph.com (release key) 2021-03-09T02:45:59.747 DEBUG:teuthology.orchestra.run.smithi144:> sudo lsb_release -is 2021-03-09T02:45:59.781 INFO:teuthology.orchestra.run.smithi092.stdout:Ubuntu 2021-03-09T02:45:59.787 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-09T02:45:59.787 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-mirror on remote deb x86_64 2021-03-09T02:45:59.788 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=9e448db9e385343da144613d840349182443aeee 2021-03-09T02:45:59.800 INFO:teuthology.orchestra.run.smithi144.stdout:Ubuntu 2021-03-09T02:45:59.806 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-09T02:45:59.806 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, rbd-mirror on remote deb x86_64 2021-03-09T02:45:59.807 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=9e448db9e385343da144613d840349182443aeee 2021-03-09T02:45:59.939 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default/ 2021-03-09T02:45:59.939 INFO:teuthology.task.install.deb:Package version is 16.1.0-627-g9e448db9-1bionic 2021-03-09T02:45:59.952 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default/ 2021-03-09T02:45:59.953 INFO:teuthology.task.install.deb:Package version is 16.1.0-627-g9e448db9-1bionic 2021-03-09T02:46:00.056 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:46:00.057 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-03-09T02:46:00.067 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:46:00.068 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2021-03-09T02:46:00.071 DEBUG:teuthology.orchestra.run.smithi092:> sudo apt-get update 2021-03-09T02:46:00.084 DEBUG:teuthology.orchestra.run.smithi144:> sudo apt-get update 2021-03-09T02:46:00.250 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-03-09T02:46:00.268 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2021-03-09T02:46:00.313 INFO:teuthology.orchestra.run.smithi092.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic InRelease 2021-03-09T02:46:00.331 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:2 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic InRelease 2021-03-09T02:46:00.336 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-03-09T02:46:00.354 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2021-03-09T02:46:00.360 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-03-09T02:46:00.361 INFO:teuthology.orchestra.run.smithi092.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic Release.gpg 2021-03-09T02:46:00.380 INFO:teuthology.orchestra.run.smithi144.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic Release.gpg 2021-03-09T02:46:00.385 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,116 B] 2021-03-09T02:46:00.386 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease 2021-03-09T02:46:00.403 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.4 kB] 2021-03-09T02:46:00.418 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.4 kB] 2021-03-09T02:46:00.436 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,116 B] 2021-03-09T02:46:00.438 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-03-09T02:46:00.464 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2021-03-09T02:46:00.517 INFO:teuthology.orchestra.run.smithi092.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-03-09T02:46:00.545 INFO:teuthology.orchestra.run.smithi144.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2021-03-09T02:46:00.830 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 27.2 kB in 1s (48.6 kB/s) 2021-03-09T02:46:00.857 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 27.2 kB in 1s (47.9 kB/s) 2021-03-09T02:46:02.067 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2021-03-09T02:46:02.101 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2021-03-09T02:46:02.122 DEBUG:teuthology.orchestra.run.smithi092:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.1.0-627-g9e448db9-1bionic cephadm=16.1.0-627-g9e448db9-1bionic ceph-mds=16.1.0-627-g9e448db9-1bionic ceph-mgr=16.1.0-627-g9e448db9-1bionic ceph-common=16.1.0-627-g9e448db9-1bionic ceph-fuse=16.1.0-627-g9e448db9-1bionic ceph-test=16.1.0-627-g9e448db9-1bionic radosgw=16.1.0-627-g9e448db9-1bionic python3-rados=16.1.0-627-g9e448db9-1bionic python3-rgw=16.1.0-627-g9e448db9-1bionic python3-cephfs=16.1.0-627-g9e448db9-1bionic python3-rbd=16.1.0-627-g9e448db9-1bionic libcephfs2=16.1.0-627-g9e448db9-1bionic libcephfs-dev=16.1.0-627-g9e448db9-1bionic librados2=16.1.0-627-g9e448db9-1bionic librbd1=16.1.0-627-g9e448db9-1bionic rbd-fuse=16.1.0-627-g9e448db9-1bionic rbd-mirror=16.1.0-627-g9e448db9-1bionic 2021-03-09T02:46:02.149 DEBUG:teuthology.orchestra.run.smithi144:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.1.0-627-g9e448db9-1bionic cephadm=16.1.0-627-g9e448db9-1bionic ceph-mds=16.1.0-627-g9e448db9-1bionic ceph-mgr=16.1.0-627-g9e448db9-1bionic ceph-common=16.1.0-627-g9e448db9-1bionic ceph-fuse=16.1.0-627-g9e448db9-1bionic ceph-test=16.1.0-627-g9e448db9-1bionic radosgw=16.1.0-627-g9e448db9-1bionic python3-rados=16.1.0-627-g9e448db9-1bionic python3-rgw=16.1.0-627-g9e448db9-1bionic python3-cephfs=16.1.0-627-g9e448db9-1bionic python3-rbd=16.1.0-627-g9e448db9-1bionic libcephfs2=16.1.0-627-g9e448db9-1bionic libcephfs-dev=16.1.0-627-g9e448db9-1bionic librados2=16.1.0-627-g9e448db9-1bionic librbd1=16.1.0-627-g9e448db9-1bionic rbd-fuse=16.1.0-627-g9e448db9-1bionic rbd-mirror=16.1.0-627-g9e448db9-1bionic 2021-03-09T02:46:02.203 INFO:teuthology.orchestra.run.smithi092.stdout:Reading package lists... 2021-03-09T02:46:02.785 INFO:teuthology.orchestra.run.smithi144.stdout:Reading package lists... 2021-03-09T02:46:02.785 INFO:teuthology.orchestra.run.smithi144.stdout:Building dependency tree... 2021-03-09T02:46:02.785 INFO:teuthology.orchestra.run.smithi144.stdout:Reading state information... 2021-03-09T02:46:02.786 INFO:teuthology.orchestra.run.smithi144.stdout:The following additional packages will be installed: 2021-03-09T02:46:02.786 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2021-03-09T02:46:02.786 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2021-03-09T02:46:02.786 INFO:teuthology.orchestra.run.smithi144.stdout: libjaeger libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-03-09T02:46:02.787 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-03-09T02:46:02.787 INFO:teuthology.orchestra.run.smithi144.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-03-09T02:46:02.787 INFO:teuthology.orchestra.run.smithi144.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-03-09T02:46:02.787 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-03-09T02:46:02.787 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2021-03-09T02:46:02.788 INFO:teuthology.orchestra.run.smithi144.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-03-09T02:46:02.788 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-03-09T02:46:02.788 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-03-09T02:46:02.788 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-03-09T02:46:02.789 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-03-09T02:46:02.789 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-03-09T02:46:02.789 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-03-09T02:46:02.789 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2021-03-09T02:46:02.789 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-03-09T02:46:02.790 INFO:teuthology.orchestra.run.smithi144.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-03-09T02:46:02.790 INFO:teuthology.orchestra.run.smithi144.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-03-09T02:46:02.790 INFO:teuthology.orchestra.run.smithi144.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-03-09T02:46:02.790 INFO:teuthology.orchestra.run.smithi144.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-03-09T02:46:02.791 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-03-09T02:46:02.791 INFO:teuthology.orchestra.run.smithi144.stdout:Suggested packages: 2021-03-09T02:46:02.791 INFO:teuthology.orchestra.run.smithi144.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-03-09T02:46:02.791 INFO:teuthology.orchestra.run.smithi144.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-03-09T02:46:02.792 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-03-09T02:46:02.792 INFO:teuthology.orchestra.run.smithi144.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-03-09T02:46:02.792 INFO:teuthology.orchestra.run.smithi144.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-03-09T02:46:02.792 INFO:teuthology.orchestra.run.smithi144.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-03-09T02:46:02.792 INFO:teuthology.orchestra.run.smithi144.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-03-09T02:46:02.793 INFO:teuthology.orchestra.run.smithi144.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-03-09T02:46:02.794 INFO:teuthology.orchestra.run.smithi144.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-03-09T02:46:02.794 INFO:teuthology.orchestra.run.smithi144.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-03-09T02:46:02.794 INFO:teuthology.orchestra.run.smithi144.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-03-09T02:46:02.794 INFO:teuthology.orchestra.run.smithi144.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-03-09T02:46:02.795 INFO:teuthology.orchestra.run.smithi144.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-03-09T02:46:02.795 INFO:teuthology.orchestra.run.smithi144.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-03-09T02:46:02.795 INFO:teuthology.orchestra.run.smithi144.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-03-09T02:46:02.795 INFO:teuthology.orchestra.run.smithi144.stdout:The following NEW packages will be installed: 2021-03-09T02:46:02.795 INFO:teuthology.orchestra.run.smithi144.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-03-09T02:46:02.796 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2021-03-09T02:46:02.796 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq 2021-03-09T02:46:02.796 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-dev libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 2021-03-09T02:46:02.796 INFO:teuthology.orchestra.run.smithi144.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-03-09T02:46:02.797 INFO:teuthology.orchestra.run.smithi144.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-03-09T02:46:02.797 INFO:teuthology.orchestra.run.smithi144.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-03-09T02:46:02.797 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-03-09T02:46:02.797 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-03-09T02:46:02.797 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2021-03-09T02:46:02.798 INFO:teuthology.orchestra.run.smithi144.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2021-03-09T02:46:02.798 INFO:teuthology.orchestra.run.smithi144.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-json-pointer 2021-03-09T02:46:02.798 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch python3-jwt python3-kubernetes python3-logutils 2021-03-09T02:46:02.798 INFO:teuthology.orchestra.run.smithi144.stdout: python3-lxml python3-mako python3-markupsafe python3-matplotlib 2021-03-09T02:46:02.799 INFO:teuthology.orchestra.run.smithi144.stdout: python3-monotonic python3-numpy python3-oauth2client python3-olefile 2021-03-09T02:46:02.799 INFO:teuthology.orchestra.run.smithi144.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2021-03-09T02:46:02.799 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan python3-pil python3-pluggy python3-prettytable python3-py 2021-03-09T02:46:02.800 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyparsing 2021-03-09T02:46:02.800 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-03-09T02:46:02.800 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-03-09T02:46:02.801 INFO:teuthology.orchestra.run.smithi144.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-03-09T02:46:02.801 INFO:teuthology.orchestra.run.smithi144.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-03-09T02:46:02.801 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-03-09T02:46:02.801 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-03-09T02:46:02.801 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug radosgw rbd-fuse rbd-mirror socat xmlstarlet 2021-03-09T02:46:02.802 INFO:teuthology.orchestra.run.smithi144.stdout:The following packages will be upgraded: 2021-03-09T02:46:02.803 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 librbd1 2021-03-09T02:46:02.805 INFO:teuthology.orchestra.run.smithi092.stdout:Building dependency tree... 2021-03-09T02:46:02.805 INFO:teuthology.orchestra.run.smithi092.stdout:Reading state information... 2021-03-09T02:46:02.805 INFO:teuthology.orchestra.run.smithi092.stdout:The following additional packages will be installed: 2021-03-09T02:46:02.805 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2021-03-09T02:46:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2021-03-09T02:46:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: libjaeger libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2021-03-09T02:46:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2021-03-09T02:46:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2021-03-09T02:46:02.806 INFO:teuthology.orchestra.run.smithi092.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-03-09T02:46:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2021-03-09T02:46:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2021-03-09T02:46:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2021-03-09T02:46:02.807 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jinja2 python3-joblib python3-json-pointer python3-jsonpatch 2021-03-09T02:46:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jwt python3-kubernetes python3-logutils python3-lxml python3-mako 2021-03-09T02:46:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2021-03-09T02:46:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2021-03-09T02:46:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2021-03-09T02:46:02.808 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2021-03-09T02:46:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2021-03-09T02:46:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2021-03-09T02:46:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2021-03-09T02:46:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2021-03-09T02:46:02.809 INFO:teuthology.orchestra.run.smithi092.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2021-03-09T02:46:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2021-03-09T02:46:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2021-03-09T02:46:02.810 INFO:teuthology.orchestra.run.smithi092.stdout:Suggested packages: 2021-03-09T02:46:02.810 INFO:teuthology.orchestra.run.smithi092.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2021-03-09T02:46:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: python-pytest-xdist python-mock python-cryptography-doc 2021-03-09T02:46:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2021-03-09T02:46:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2021-03-09T02:46:02.811 INFO:teuthology.orchestra.run.smithi092.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2021-03-09T02:46:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2021-03-09T02:46:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2021-03-09T02:46:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2021-03-09T02:46:02.812 INFO:teuthology.orchestra.run.smithi092.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2021-03-09T02:46:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2021-03-09T02:46:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2021-03-09T02:46:02.813 INFO:teuthology.orchestra.run.smithi092.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2021-03-09T02:46:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2021-03-09T02:46:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2021-03-09T02:46:02.814 INFO:teuthology.orchestra.run.smithi092.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2021-03-09T02:46:02.814 INFO:teuthology.orchestra.run.smithi092.stdout:The following NEW packages will be installed: 2021-03-09T02:46:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2021-03-09T02:46:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2021-03-09T02:46:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm jq 2021-03-09T02:46:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: libcephfs-dev libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 2021-03-09T02:46:02.815 INFO:teuthology.orchestra.run.smithi092.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2021-03-09T02:46:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2021-03-09T02:46:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2021-03-09T02:46:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2021-03-09T02:46:02.816 INFO:teuthology.orchestra.run.smithi092.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2021-03-09T02:46:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2021-03-09T02:46:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2021-03-09T02:46:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-json-pointer 2021-03-09T02:46:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: python3-jsonpatch python3-jwt python3-kubernetes python3-logutils 2021-03-09T02:46:02.817 INFO:teuthology.orchestra.run.smithi092.stdout: python3-lxml python3-mako python3-markupsafe python3-matplotlib 2021-03-09T02:46:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: python3-monotonic python3-numpy python3-oauth2client python3-olefile 2021-03-09T02:46:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2021-03-09T02:46:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pecan python3-pil python3-pluggy python3-prettytable python3-py 2021-03-09T02:46:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pyasn1 python3-pyasn1-modules python3-pyinotify python3-pyparsing 2021-03-09T02:46:02.818 INFO:teuthology.orchestra.run.smithi092.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2021-03-09T02:46:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2021-03-09T02:46:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2021-03-09T02:46:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2021-03-09T02:46:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2021-03-09T02:46:02.819 INFO:teuthology.orchestra.run.smithi092.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2021-03-09T02:46:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: python3-werkzeug radosgw rbd-fuse rbd-mirror socat xmlstarlet 2021-03-09T02:46:02.820 INFO:teuthology.orchestra.run.smithi092.stdout:The following packages will be upgraded: 2021-03-09T02:46:02.820 INFO:teuthology.orchestra.run.smithi092.stdout: librados2 librbd1 2021-03-09T02:46:02.864 INFO:teuthology.orchestra.run.smithi092.stdout:2 upgraded, 112 newly installed, 0 to remove and 15 not upgraded. 2021-03-09T02:46:02.864 INFO:teuthology.orchestra.run.smithi092.stdout:Need to get 168 MB of archives. 2021-03-09T02:46:02.864 INFO:teuthology.orchestra.run.smithi092.stdout:After this operation, 786 MB of additional disk space will be used. 2021-03-09T02:46:02.865 INFO:teuthology.orchestra.run.smithi092.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 16.1.0-627-g9e448db9-1bionic [3,720 B] 2021-03-09T02:46:02.910 INFO:teuthology.orchestra.run.smithi092.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.1.0-627-g9e448db9-1bionic [3,119 kB] 2021-03-09T02:46:02.919 INFO:teuthology.orchestra.run.smithi092.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-03-09T02:46:02.920 INFO:teuthology.orchestra.run.smithi144.stdout:2 upgraded, 112 newly installed, 0 to remove and 15 not upgraded. 2021-03-09T02:46:02.920 INFO:teuthology.orchestra.run.smithi144.stdout:Need to get 168 MB of archives. 2021-03-09T02:46:02.920 INFO:teuthology.orchestra.run.smithi144.stdout:After this operation, 786 MB of additional disk space will be used. 2021-03-09T02:46:02.921 INFO:teuthology.orchestra.run.smithi144.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 16.1.0-627-g9e448db9-1bionic [3,720 B] 2021-03-09T02:46:02.966 INFO:teuthology.orchestra.run.smithi144.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.1.0-627-g9e448db9-1bionic [3,119 kB] 2021-03-09T02:46:02.972 INFO:teuthology.orchestra.run.smithi144.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2021-03-09T02:46:03.001 INFO:teuthology.orchestra.run.smithi092.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-03-09T02:46:03.052 INFO:teuthology.orchestra.run.smithi144.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2021-03-09T02:46:03.260 INFO:teuthology.orchestra.run.smithi092.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.1.0-627-g9e448db9-1bionic [3,164 kB] 2021-03-09T02:46:03.262 INFO:teuthology.orchestra.run.smithi092.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-03-09T02:46:03.277 INFO:teuthology.orchestra.run.smithi092.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-03-09T02:46:03.297 INFO:teuthology.orchestra.run.smithi092.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-03-09T02:46:03.307 INFO:teuthology.orchestra.run.smithi144.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2021-03-09T02:46:03.313 INFO:teuthology.orchestra.run.smithi144.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.1.0-627-g9e448db9-1bionic [3,164 kB] 2021-03-09T02:46:03.322 INFO:teuthology.orchestra.run.smithi144.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2021-03-09T02:46:03.336 INFO:teuthology.orchestra.run.smithi144.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2021-03-09T02:46:03.398 INFO:teuthology.orchestra.run.smithi092.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-03-09T02:46:03.402 INFO:teuthology.orchestra.run.smithi092.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-03-09T02:46:03.426 INFO:teuthology.orchestra.run.smithi092.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-03-09T02:46:03.432 INFO:teuthology.orchestra.run.smithi144.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2021-03-09T02:46:03.433 INFO:teuthology.orchestra.run.smithi092.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-03-09T02:46:03.436 INFO:teuthology.orchestra.run.smithi144.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2021-03-09T02:46:03.441 INFO:teuthology.orchestra.run.smithi092.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-03-09T02:46:03.452 INFO:teuthology.orchestra.run.smithi092.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-03-09T02:46:03.464 INFO:teuthology.orchestra.run.smithi092.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-03-09T02:46:03.464 INFO:teuthology.orchestra.run.smithi092.stdout:Get:16 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.1.0-627-g9e448db9-1bionic [664 kB] 2021-03-09T02:46:03.479 INFO:teuthology.orchestra.run.smithi144.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2021-03-09T02:46:03.486 INFO:teuthology.orchestra.run.smithi144.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2021-03-09T02:46:03.488 INFO:teuthology.orchestra.run.smithi092.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-03-09T02:46:03.493 INFO:teuthology.orchestra.run.smithi092.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-03-09T02:46:03.494 INFO:teuthology.orchestra.run.smithi144.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2021-03-09T02:46:03.500 INFO:teuthology.orchestra.run.smithi092.stdout:Get:19 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.1.0-627-g9e448db9-1bionic [340 kB] 2021-03-09T02:46:03.503 INFO:teuthology.orchestra.run.smithi144.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2021-03-09T02:46:03.509 INFO:teuthology.orchestra.run.smithi144.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2021-03-09T02:46:03.510 INFO:teuthology.orchestra.run.smithi092.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-03-09T02:46:03.523 INFO:teuthology.orchestra.run.smithi092.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-03-09T02:46:03.524 INFO:teuthology.orchestra.run.smithi092.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-03-09T02:46:03.527 INFO:teuthology.orchestra.run.smithi092.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.1.0-627-g9e448db9-1bionic [21.7 kB] 2021-03-09T02:46:03.528 INFO:teuthology.orchestra.run.smithi092.stdout:Get:24 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.1.0-627-g9e448db9-1bionic [175 kB] 2021-03-09T02:46:03.529 INFO:teuthology.orchestra.run.smithi144.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2021-03-09T02:46:03.533 INFO:teuthology.orchestra.run.smithi144.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2021-03-09T02:46:03.534 INFO:teuthology.orchestra.run.smithi092.stdout:Get:25 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.1.0-627-g9e448db9-1bionic [29.3 kB] 2021-03-09T02:46:03.537 INFO:teuthology.orchestra.run.smithi092.stdout:Get:26 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.1.0-627-g9e448db9-1bionic [336 kB] 2021-03-09T02:46:03.544 INFO:teuthology.orchestra.run.smithi144.stdout:Get:18 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.1.0-627-g9e448db9-1bionic [664 kB] 2021-03-09T02:46:03.558 INFO:teuthology.orchestra.run.smithi144.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2021-03-09T02:46:03.563 INFO:teuthology.orchestra.run.smithi092.stdout:Get:27 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.1.0-627-g9e448db9-1bionic [3,234 kB] 2021-03-09T02:46:03.570 INFO:teuthology.orchestra.run.smithi144.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2021-03-09T02:46:03.585 INFO:teuthology.orchestra.run.smithi144.stdout:Get:21 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.1.0-627-g9e448db9-1bionic [340 kB] 2021-03-09T02:46:03.590 INFO:teuthology.orchestra.run.smithi092.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-03-09T02:46:03.591 INFO:teuthology.orchestra.run.smithi092.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-03-09T02:46:03.593 INFO:teuthology.orchestra.run.smithi092.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-03-09T02:46:03.594 INFO:teuthology.orchestra.run.smithi092.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-03-09T02:46:03.594 INFO:teuthology.orchestra.run.smithi092.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-03-09T02:46:03.598 INFO:teuthology.orchestra.run.smithi092.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-03-09T02:46:03.600 INFO:teuthology.orchestra.run.smithi092.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-03-09T02:46:03.613 INFO:teuthology.orchestra.run.smithi144.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.1.0-627-g9e448db9-1bionic [21.7 kB] 2021-03-09T02:46:03.616 INFO:teuthology.orchestra.run.smithi144.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.1.0-627-g9e448db9-1bionic [175 kB] 2021-03-09T02:46:03.622 INFO:teuthology.orchestra.run.smithi144.stdout:Get:24 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.1.0-627-g9e448db9-1bionic [29.3 kB] 2021-03-09T02:46:03.623 INFO:teuthology.orchestra.run.smithi144.stdout:Get:25 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.1.0-627-g9e448db9-1bionic [336 kB] 2021-03-09T02:46:03.638 INFO:teuthology.orchestra.run.smithi144.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2021-03-09T02:46:03.638 INFO:teuthology.orchestra.run.smithi144.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2021-03-09T02:46:03.639 INFO:teuthology.orchestra.run.smithi144.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2021-03-09T02:46:03.641 INFO:teuthology.orchestra.run.smithi144.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2021-03-09T02:46:03.642 INFO:teuthology.orchestra.run.smithi144.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2021-03-09T02:46:03.643 INFO:teuthology.orchestra.run.smithi144.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2021-03-09T02:46:03.645 INFO:teuthology.orchestra.run.smithi144.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2021-03-09T02:46:03.647 INFO:teuthology.orchestra.run.smithi144.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2021-03-09T02:46:03.653 INFO:teuthology.orchestra.run.smithi144.stdout:Get:34 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.1.0-627-g9e448db9-1bionic [3,234 kB] 2021-03-09T02:46:03.671 INFO:teuthology.orchestra.run.smithi092.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-03-09T02:46:03.672 INFO:teuthology.orchestra.run.smithi092.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-03-09T02:46:03.686 INFO:teuthology.orchestra.run.smithi092.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-03-09T02:46:03.717 INFO:teuthology.orchestra.run.smithi144.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2021-03-09T02:46:03.718 INFO:teuthology.orchestra.run.smithi144.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2021-03-09T02:46:03.752 INFO:teuthology.orchestra.run.smithi092.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-03-09T02:46:03.753 INFO:teuthology.orchestra.run.smithi092.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-03-09T02:46:03.754 INFO:teuthology.orchestra.run.smithi092.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-03-09T02:46:03.758 INFO:teuthology.orchestra.run.smithi092.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-03-09T02:46:03.759 INFO:teuthology.orchestra.run.smithi092.stdout:Get:42 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.1.0-627-g9e448db9-1bionic [99.2 kB] 2021-03-09T02:46:03.763 INFO:teuthology.orchestra.run.smithi092.stdout:Get:43 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.1.0-627-g9e448db9-1bionic [415 kB] 2021-03-09T02:46:03.764 INFO:teuthology.orchestra.run.smithi092.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-03-09T02:46:03.766 INFO:teuthology.orchestra.run.smithi092.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-03-09T02:46:03.767 INFO:teuthology.orchestra.run.smithi092.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-03-09T02:46:03.796 INFO:teuthology.orchestra.run.smithi092.stdout:Get:47 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.1.0-627-g9e448db9-1bionic [21.0 MB] 2021-03-09T02:46:03.797 INFO:teuthology.orchestra.run.smithi144.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2021-03-09T02:46:03.797 INFO:teuthology.orchestra.run.smithi144.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2021-03-09T02:46:03.798 INFO:teuthology.orchestra.run.smithi144.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2021-03-09T02:46:03.799 INFO:teuthology.orchestra.run.smithi144.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2021-03-09T02:46:03.804 INFO:teuthology.orchestra.run.smithi144.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.1 [174 kB] 2021-03-09T02:46:03.814 INFO:teuthology.orchestra.run.smithi144.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2021-03-09T02:46:03.814 INFO:teuthology.orchestra.run.smithi144.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2021-03-09T02:46:03.815 INFO:teuthology.orchestra.run.smithi144.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2021-03-09T02:46:03.833 INFO:teuthology.orchestra.run.smithi092.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-03-09T02:46:03.872 INFO:teuthology.orchestra.run.smithi144.stdout:Get:45 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.1.0-627-g9e448db9-1bionic [99.2 kB] 2021-03-09T02:46:03.875 INFO:teuthology.orchestra.run.smithi144.stdout:Get:46 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.1.0-627-g9e448db9-1bionic [415 kB] 2021-03-09T02:46:03.876 INFO:teuthology.orchestra.run.smithi144.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2021-03-09T02:46:03.915 INFO:teuthology.orchestra.run.smithi144.stdout:Get:48 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.1.0-627-g9e448db9-1bionic [21.0 MB] 2021-03-09T02:46:03.921 INFO:teuthology.orchestra.run.smithi092.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-03-09T02:46:03.921 INFO:teuthology.orchestra.run.smithi092.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-03-09T02:46:03.975 INFO:teuthology.orchestra.run.smithi144.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2021-03-09T02:46:03.975 INFO:teuthology.orchestra.run.smithi144.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2021-03-09T02:46:04.188 INFO:teuthology.orchestra.run.smithi092.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-03-09T02:46:04.221 INFO:teuthology.orchestra.run.smithi092.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-03-09T02:46:04.237 INFO:teuthology.orchestra.run.smithi092.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-03-09T02:46:04.237 INFO:teuthology.orchestra.run.smithi144.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2021-03-09T02:46:04.273 INFO:teuthology.orchestra.run.smithi092.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-03-09T02:46:04.274 INFO:teuthology.orchestra.run.smithi092.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-03-09T02:46:04.274 INFO:teuthology.orchestra.run.smithi092.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2021-03-09T02:46:04.276 INFO:teuthology.orchestra.run.smithi092.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-03-09T02:46:04.276 INFO:teuthology.orchestra.run.smithi092.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-03-09T02:46:04.277 INFO:teuthology.orchestra.run.smithi092.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-03-09T02:46:04.277 INFO:teuthology.orchestra.run.smithi092.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-03-09T02:46:04.305 INFO:teuthology.orchestra.run.smithi144.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2021-03-09T02:46:04.306 INFO:teuthology.orchestra.run.smithi144.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2021-03-09T02:46:04.319 INFO:teuthology.orchestra.run.smithi144.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2021-03-09T02:46:04.319 INFO:teuthology.orchestra.run.smithi144.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2021-03-09T02:46:04.320 INFO:teuthology.orchestra.run.smithi144.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2021-03-09T02:46:04.322 INFO:teuthology.orchestra.run.smithi144.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2021-03-09T02:46:04.322 INFO:teuthology.orchestra.run.smithi144.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2021-03-09T02:46:04.322 INFO:teuthology.orchestra.run.smithi144.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2021-03-09T02:46:04.323 INFO:teuthology.orchestra.run.smithi144.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2021-03-09T02:46:04.350 INFO:teuthology.orchestra.run.smithi092.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-03-09T02:46:04.351 INFO:teuthology.orchestra.run.smithi092.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-03-09T02:46:04.396 INFO:teuthology.orchestra.run.smithi144.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2021-03-09T02:46:04.399 INFO:teuthology.orchestra.run.smithi144.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2021-03-09T02:46:04.431 INFO:teuthology.orchestra.run.smithi092.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-03-09T02:46:04.432 INFO:teuthology.orchestra.run.smithi092.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-03-09T02:46:04.436 INFO:teuthology.orchestra.run.smithi092.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-03-09T02:46:04.437 INFO:teuthology.orchestra.run.smithi092.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-03-09T02:46:04.437 INFO:teuthology.orchestra.run.smithi092.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-03-09T02:46:04.440 INFO:teuthology.orchestra.run.smithi092.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-03-09T02:46:04.442 INFO:teuthology.orchestra.run.smithi092.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-03-09T02:46:04.443 INFO:teuthology.orchestra.run.smithi092.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-03-09T02:46:04.475 INFO:teuthology.orchestra.run.smithi144.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2021-03-09T02:46:04.476 INFO:teuthology.orchestra.run.smithi144.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2021-03-09T02:46:04.481 INFO:teuthology.orchestra.run.smithi144.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2021-03-09T02:46:04.481 INFO:teuthology.orchestra.run.smithi144.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2021-03-09T02:46:04.482 INFO:teuthology.orchestra.run.smithi144.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2021-03-09T02:46:04.485 INFO:teuthology.orchestra.run.smithi144.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2021-03-09T02:46:04.487 INFO:teuthology.orchestra.run.smithi144.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2021-03-09T02:46:04.489 INFO:teuthology.orchestra.run.smithi144.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2021-03-09T02:46:04.512 INFO:teuthology.orchestra.run.smithi092.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-03-09T02:46:04.516 INFO:teuthology.orchestra.run.smithi092.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-03-09T02:46:04.556 INFO:teuthology.orchestra.run.smithi144.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2021-03-09T02:46:04.560 INFO:teuthology.orchestra.run.smithi144.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2021-03-09T02:46:04.593 INFO:teuthology.orchestra.run.smithi092.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-03-09T02:46:04.599 INFO:teuthology.orchestra.run.smithi092.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-03-09T02:46:04.599 INFO:teuthology.orchestra.run.smithi092.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-03-09T02:46:04.600 INFO:teuthology.orchestra.run.smithi092.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-03-09T02:46:04.601 INFO:teuthology.orchestra.run.smithi092.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-03-09T02:46:04.602 INFO:teuthology.orchestra.run.smithi092.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-03-09T02:46:04.604 INFO:teuthology.orchestra.run.smithi092.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-03-09T02:46:04.605 INFO:teuthology.orchestra.run.smithi092.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-03-09T02:46:04.634 INFO:teuthology.orchestra.run.smithi144.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.1 [184 kB] 2021-03-09T02:46:04.641 INFO:teuthology.orchestra.run.smithi144.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2021-03-09T02:46:04.642 INFO:teuthology.orchestra.run.smithi144.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2021-03-09T02:46:04.643 INFO:teuthology.orchestra.run.smithi144.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2021-03-09T02:46:04.643 INFO:teuthology.orchestra.run.smithi144.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2021-03-09T02:46:04.645 INFO:teuthology.orchestra.run.smithi144.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2021-03-09T02:46:04.647 INFO:teuthology.orchestra.run.smithi144.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2021-03-09T02:46:04.648 INFO:teuthology.orchestra.run.smithi144.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2021-03-09T02:46:04.673 INFO:teuthology.orchestra.run.smithi092.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-03-09T02:46:04.674 INFO:teuthology.orchestra.run.smithi092.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-03-09T02:46:04.713 INFO:teuthology.orchestra.run.smithi144.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2021-03-09T02:46:04.714 INFO:teuthology.orchestra.run.smithi144.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2021-03-09T02:46:04.754 INFO:teuthology.orchestra.run.smithi092.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-03-09T02:46:04.782 INFO:teuthology.orchestra.run.smithi092.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-03-09T02:46:04.783 INFO:teuthology.orchestra.run.smithi092.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-03-09T02:46:04.784 INFO:teuthology.orchestra.run.smithi092.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-03-09T02:46:04.793 INFO:teuthology.orchestra.run.smithi144.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.3 [1,097 kB] 2021-03-09T02:46:04.816 INFO:teuthology.orchestra.run.smithi144.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2021-03-09T02:46:04.816 INFO:teuthology.orchestra.run.smithi144.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2021-03-09T02:46:04.816 INFO:teuthology.orchestra.run.smithi144.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2021-03-09T02:46:04.875 INFO:teuthology.orchestra.run.smithi092.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-03-09T02:46:04.876 INFO:teuthology.orchestra.run.smithi092.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-03-09T02:46:04.877 INFO:teuthology.orchestra.run.smithi092.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.4 [329 kB] 2021-03-09T02:46:04.920 INFO:teuthology.orchestra.run.smithi092.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-03-09T02:46:04.920 INFO:teuthology.orchestra.run.smithi092.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-03-09T02:46:04.921 INFO:teuthology.orchestra.run.smithi092.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-03-09T02:46:04.922 INFO:teuthology.orchestra.run.smithi092.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-03-09T02:46:04.925 INFO:teuthology.orchestra.run.smithi092.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-03-09T02:46:04.926 INFO:teuthology.orchestra.run.smithi092.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-03-09T02:46:04.953 INFO:teuthology.orchestra.run.smithi144.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2021-03-09T02:46:04.954 INFO:teuthology.orchestra.run.smithi144.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2021-03-09T02:46:04.955 INFO:teuthology.orchestra.run.smithi144.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.4 [329 kB] 2021-03-09T02:46:04.962 INFO:teuthology.orchestra.run.smithi144.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2021-03-09T02:46:04.962 INFO:teuthology.orchestra.run.smithi144.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2021-03-09T02:46:04.963 INFO:teuthology.orchestra.run.smithi144.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2021-03-09T02:46:04.963 INFO:teuthology.orchestra.run.smithi144.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2021-03-09T02:46:04.964 INFO:teuthology.orchestra.run.smithi144.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2021-03-09T02:46:04.965 INFO:teuthology.orchestra.run.smithi144.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2021-03-09T02:46:05.125 INFO:teuthology.orchestra.run.smithi092.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.1.0-627-g9e448db9-1bionic [5,634 kB] 2021-03-09T02:46:05.344 INFO:teuthology.orchestra.run.smithi144.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.1.0-627-g9e448db9-1bionic [5,634 kB] 2021-03-09T02:46:05.493 INFO:teuthology.orchestra.run.smithi092.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.1.0-627-g9e448db9-1bionic [183 kB] 2021-03-09T02:46:05.499 INFO:teuthology.orchestra.run.smithi092.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.1.0-627-g9e448db9-1bionic [1,388 kB] 2021-03-09T02:46:05.575 INFO:teuthology.orchestra.run.smithi092.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.1.0-627-g9e448db9-1bionic [6,671 kB] 2021-03-09T02:46:05.735 INFO:teuthology.orchestra.run.smithi144.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.1.0-627-g9e448db9-1bionic [183 kB] 2021-03-09T02:46:05.740 INFO:teuthology.orchestra.run.smithi144.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.1.0-627-g9e448db9-1bionic [1,388 kB] 2021-03-09T02:46:05.821 INFO:teuthology.orchestra.run.smithi144.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.1.0-627-g9e448db9-1bionic [6,671 kB] 2021-03-09T02:46:06.024 INFO:teuthology.orchestra.run.smithi092.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.1.0-627-g9e448db9-1bionic [24.9 MB] 2021-03-09T02:46:06.226 INFO:teuthology.orchestra.run.smithi144.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.1.0-627-g9e448db9-1bionic [24.9 MB] 2021-03-09T02:46:07.666 INFO:teuthology.orchestra.run.smithi144.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.1.0-627-g9e448db9-1bionic [3,808 B] 2021-03-09T02:46:07.666 INFO:teuthology.orchestra.run.smithi144.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.1.0-627-g9e448db9-1bionic [770 kB] 2021-03-09T02:46:07.691 INFO:teuthology.orchestra.run.smithi144.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.1.0-627-g9e448db9-1bionic [2,153 kB] 2021-03-09T02:46:07.818 INFO:teuthology.orchestra.run.smithi144.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.1.0-627-g9e448db9-1bionic [62.2 kB] 2021-03-09T02:46:07.822 INFO:teuthology.orchestra.run.smithi144.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.1.0-627-g9e448db9-1bionic [67.9 kB] 2021-03-09T02:46:07.823 INFO:teuthology.orchestra.run.smithi144.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.1.0-627-g9e448db9-1bionic [2,329 kB] 2021-03-09T02:46:07.924 INFO:teuthology.orchestra.run.smithi092.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.1.0-627-g9e448db9-1bionic [3,808 B] 2021-03-09T02:46:07.925 INFO:teuthology.orchestra.run.smithi092.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.1.0-627-g9e448db9-1bionic [770 kB] 2021-03-09T02:46:07.955 INFO:teuthology.orchestra.run.smithi144.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.1.0-627-g9e448db9-1bionic [7,862 kB] 2021-03-09T02:46:07.984 INFO:teuthology.orchestra.run.smithi092.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.1.0-627-g9e448db9-1bionic [2,153 kB] 2021-03-09T02:46:08.169 INFO:teuthology.orchestra.run.smithi092.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.1.0-627-g9e448db9-1bionic [62.2 kB] 2021-03-09T02:46:08.171 INFO:teuthology.orchestra.run.smithi092.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.1.0-627-g9e448db9-1bionic [67.9 kB] 2021-03-09T02:46:08.174 INFO:teuthology.orchestra.run.smithi092.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.1.0-627-g9e448db9-1bionic [2,329 kB] 2021-03-09T02:46:08.369 INFO:teuthology.orchestra.run.smithi092.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.1.0-627-g9e448db9-1bionic [7,862 kB] 2021-03-09T02:46:08.414 INFO:teuthology.orchestra.run.smithi144.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.1.0-627-g9e448db9-1bionic [3,928 B] 2021-03-09T02:46:08.414 INFO:teuthology.orchestra.run.smithi144.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.1.0-627-g9e448db9-1bionic [22.7 kB] 2021-03-09T02:46:08.414 INFO:teuthology.orchestra.run.smithi144.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.1.0-627-g9e448db9-1bionic [44.0 MB] 2021-03-09T02:46:08.947 INFO:teuthology.orchestra.run.smithi092.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.1.0-627-g9e448db9-1bionic [3,928 B] 2021-03-09T02:46:08.948 INFO:teuthology.orchestra.run.smithi092.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.1.0-627-g9e448db9-1bionic [22.7 kB] 2021-03-09T02:46:08.948 INFO:teuthology.orchestra.run.smithi092.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.1.0-627-g9e448db9-1bionic [44.0 MB] 2021-03-09T02:46:10.767 INFO:teuthology.orchestra.run.smithi144.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.1.0-627-g9e448db9-1bionic [19.8 kB] 2021-03-09T02:46:10.768 INFO:teuthology.orchestra.run.smithi144.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.1.0-627-g9e448db9-1bionic [9,907 kB] 2021-03-09T02:46:11.274 INFO:teuthology.orchestra.run.smithi144.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.1.0-627-g9e448db9-1bionic [77.6 kB] 2021-03-09T02:46:11.276 INFO:teuthology.orchestra.run.smithi144.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 rbd-mirror amd64 16.1.0-627-g9e448db9-1bionic [3,324 kB] 2021-03-09T02:46:11.981 INFO:teuthology.orchestra.run.smithi092.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.1.0-627-g9e448db9-1bionic [19.8 kB] 2021-03-09T02:46:11.981 INFO:teuthology.orchestra.run.smithi092.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.1.0-627-g9e448db9-1bionic [9,907 kB] 2021-03-09T02:46:12.147 INFO:teuthology.orchestra.run.smithi144.stdout:Fetched 168 MB in 9s (19.4 MB/s) 2021-03-09T02:46:12.300 INFO:teuthology.orchestra.run.smithi092.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.1.0-627-g9e448db9-1bionic [77.6 kB] 2021-03-09T02:46:12.302 INFO:teuthology.orchestra.run.smithi092.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-yuri5-testing-2021-03-08-1049-pacific/9e448db9e385343da144613d840349182443aeee/ubuntu/bionic/flavors/default bionic/main amd64 rbd-mirror amd64 16.1.0-627-g9e448db9-1bionic [3,324 kB] 2021-03-09T02:46:12.309 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-uritemplate. 2021-03-09T02:46:12.346 INFO:teuthology.orchestra.run.smithi144.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102065 files and directories currently installed.) 2021-03-09T02:46:12.349 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-03-09T02:46:12.391 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-03-09T02:46:12.618 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libjaeger. 2021-03-09T02:46:12.632 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../001-libjaeger_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:12.669 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libjaeger (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:12.971 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-03-09T02:46:12.983 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-03-09T02:46:13.063 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-03-09T02:46:13.094 INFO:teuthology.orchestra.run.smithi092.stdout:Fetched 168 MB in 10s (17.4 MB/s) 2021-03-09T02:46:13.236 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-uritemplate. 2021-03-09T02:46:13.268 INFO:teuthology.orchestra.run.smithi092.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 102065 files and directories currently installed.) 2021-03-09T02:46:13.271 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2021-03-09T02:46:13.308 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2021-03-09T02:46:13.394 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../003-librbd1_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:13.516 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librbd1 (16.1.0-627-g9e448db9-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-03-09T02:46:13.544 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libjaeger. 2021-03-09T02:46:13.557 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../001-libjaeger_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:13.594 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libjaeger (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:13.854 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2021-03-09T02:46:13.866 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2021-03-09T02:46:13.946 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2021-03-09T02:46:14.324 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../004-librados2_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:14.446 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librados2 (16.1.0-627-g9e448db9-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-03-09T02:46:14.496 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../003-librbd1_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:14.617 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librbd1 (16.1.0-627-g9e448db9-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-03-09T02:46:15.092 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs2. 2021-03-09T02:46:15.103 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../005-libcephfs2_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:15.141 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs2 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:15.291 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../004-librados2_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:15.396 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librados2 (16.1.0-627-g9e448db9-1bionic) over (12.2.13-0ubuntu0.18.04.6) ... 2021-03-09T02:46:15.444 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rados. 2021-03-09T02:46:15.456 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../006-python3-rados_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:15.485 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rados (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:15.830 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-03-09T02:46:15.841 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../007-python3-ceph-argparse_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:15.881 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-argparse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:16.016 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libcephfs2. 2021-03-09T02:46:16.028 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../005-libcephfs2_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:16.066 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libcephfs2 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:16.201 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cephfs. 2021-03-09T02:46:16.211 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../008-python3-cephfs_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:16.249 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cephfs (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:16.418 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rados. 2021-03-09T02:46:16.429 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../006-python3-rados_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:16.468 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rados (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:16.560 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-ceph-common. 2021-03-09T02:46:16.572 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../009-python3-ceph-common_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:16.610 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-ceph-common (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:16.787 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-ceph-argparse. 2021-03-09T02:46:16.799 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../007-python3-ceph-argparse_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:16.837 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-ceph-argparse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:16.896 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-prettytable. 2021-03-09T02:46:16.906 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2021-03-09T02:46:16.946 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-03-09T02:46:17.106 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cephfs. 2021-03-09T02:46:17.117 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../008-python3-cephfs_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:17.156 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cephfs (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:17.174 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rbd. 2021-03-09T02:46:17.186 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../011-python3-rbd_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:17.224 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rbd (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:17.501 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-ceph-common. 2021-03-09T02:46:17.510 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../009-python3-ceph-common_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:17.551 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-ceph-common (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:17.634 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-03-09T02:46:17.646 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../012-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-03-09T02:46:17.685 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-03-09T02:46:17.845 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-prettytable. 2021-03-09T02:46:17.858 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2021-03-09T02:46:17.895 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2021-03-09T02:46:18.013 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-03-09T02:46:18.023 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../013-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-03-09T02:46:18.063 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-03-09T02:46:18.139 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rbd. 2021-03-09T02:46:18.151 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../011-python3-rbd_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:18.190 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rbd (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:18.365 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package librgw2. 2021-03-09T02:46:18.379 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../014-librgw2_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:18.415 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking librgw2 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:18.500 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librabbitmq4:amd64. 2021-03-09T02:46:18.512 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../012-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2021-03-09T02:46:18.541 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-03-09T02:46:18.902 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librdkafka1:amd64. 2021-03-09T02:46:18.914 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../013-librdkafka1_0.11.3-1build1_amd64.deb ... 2021-03-09T02:46:18.952 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2021-03-09T02:46:18.984 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rgw. 2021-03-09T02:46:18.996 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../015-python3-rgw_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:19.034 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rgw (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:19.438 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package librgw2. 2021-03-09T02:46:19.445 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-03-09T02:46:19.450 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../014-librgw2_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:19.458 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../016-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-03-09T02:46:19.488 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking librgw2 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:19.495 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-03-09T02:46:19.739 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-03-09T02:46:19.750 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../017-libleveldb1v5_1.20-2_amd64.deb ... 2021-03-09T02:46:19.781 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-03-09T02:46:20.034 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rgw. 2021-03-09T02:46:20.043 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../015-python3-rgw_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:20.074 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rgw (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:20.200 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package liboath0. 2021-03-09T02:46:20.212 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../018-liboath0_2.6.1-1_amd64.deb ... 2021-03-09T02:46:20.250 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-03-09T02:46:20.368 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2021-03-09T02:46:20.380 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../016-libsnappy1v5_1.1.7-1_amd64.deb ... 2021-03-09T02:46:20.418 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2021-03-09T02:46:20.536 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libradosstriper1. 2021-03-09T02:46:20.547 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../019-libradosstriper1_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:20.586 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libradosstriper1 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:20.720 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2021-03-09T02:46:20.733 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../017-libleveldb1v5_1.20-2_amd64.deb ... 2021-03-09T02:46:20.795 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2021-03-09T02:46:20.981 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-common. 2021-03-09T02:46:20.993 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../020-ceph-common_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:21.023 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-common (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:21.131 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package liboath0. 2021-03-09T02:46:21.143 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../018-liboath0_2.6.1-1_amd64.deb ... 2021-03-09T02:46:21.181 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking liboath0 (2.6.1-1) ... 2021-03-09T02:46:21.492 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libradosstriper1. 2021-03-09T02:46:21.504 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../019-libradosstriper1_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:21.542 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libradosstriper1 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:22.020 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-common. 2021-03-09T02:46:22.034 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../020-ceph-common_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:22.070 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-common (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:22.953 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-base. 2021-03-09T02:46:22.965 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../021-ceph-base_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:23.021 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-base (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:23.949 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-dateutil. 2021-03-09T02:46:23.961 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../022-python3-dateutil_2.6.1-1_all.deb ... 2021-03-09T02:46:23.967 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-base. 2021-03-09T02:46:23.978 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../021-ceph-base_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:23.999 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-03-09T02:46:24.412 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-base (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:24.413 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-asn1crypto. 2021-03-09T02:46:24.413 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../023-python3-asn1crypto_0.24.0-1_all.deb ... 2021-03-09T02:46:24.413 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-03-09T02:46:24.633 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cffi-backend. 2021-03-09T02:46:24.643 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../024-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-03-09T02:46:24.680 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-03-09T02:46:24.971 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-dateutil. 2021-03-09T02:46:24.983 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../022-python3-dateutil_2.6.1-1_all.deb ... 2021-03-09T02:46:25.016 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cryptography. 2021-03-09T02:46:25.021 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2021-03-09T02:46:25.027 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../025-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-03-09T02:46:25.066 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-03-09T02:46:25.349 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-asn1crypto. 2021-03-09T02:46:25.361 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../023-python3-asn1crypto_0.24.0-1_all.deb ... 2021-03-09T02:46:25.399 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2021-03-09T02:46:25.436 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-openssl. 2021-03-09T02:46:25.448 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../026-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-03-09T02:46:25.486 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-03-09T02:46:25.653 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cffi-backend. 2021-03-09T02:46:25.662 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../024-python3-cffi-backend_1.11.5-1_amd64.deb ... 2021-03-09T02:46:25.701 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2021-03-09T02:46:25.747 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-03-09T02:46:25.759 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:25.789 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-modules-core (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:26.020 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cryptography. 2021-03-09T02:46:26.032 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../025-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2021-03-09T02:46:26.070 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-03-09T02:46:26.143 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-bcrypt. 2021-03-09T02:46:26.155 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-03-09T02:46:26.218 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-03-09T02:46:26.415 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-openssl. 2021-03-09T02:46:26.428 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../026-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2021-03-09T02:46:26.465 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2021-03-09T02:46:26.521 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cherrypy3. 2021-03-09T02:46:26.532 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-2_all.deb ... 2021-03-09T02:46:26.570 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-03-09T02:46:26.734 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2021-03-09T02:46:26.747 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:26.784 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-modules-core (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:26.915 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jwt. 2021-03-09T02:46:26.927 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../030-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-03-09T02:46:26.965 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-03-09T02:46:27.147 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-bcrypt. 2021-03-09T02:46:27.158 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.4-2_amd64.deb ... 2021-03-09T02:46:27.197 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2021-03-09T02:46:27.235 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-logutils. 2021-03-09T02:46:27.249 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../031-python3-logutils_0.3.3-5_all.deb ... 2021-03-09T02:46:27.276 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-03-09T02:46:27.524 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cherrypy3. 2021-03-09T02:46:27.536 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-2_all.deb ... 2021-03-09T02:46:27.574 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2021-03-09T02:46:27.613 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-markupsafe. 2021-03-09T02:46:27.624 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../032-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-03-09T02:46:27.663 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-03-09T02:46:27.911 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jwt. 2021-03-09T02:46:27.923 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../030-python3-jwt_1.5.3+ds1-1_all.deb ... 2021-03-09T02:46:27.960 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2021-03-09T02:46:27.965 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-mako. 2021-03-09T02:46:27.977 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../033-python3-mako_1.0.7+ds1-1_all.deb ... 2021-03-09T02:46:28.015 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-03-09T02:46:28.272 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-logutils. 2021-03-09T02:46:28.286 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../031-python3-logutils_0.3.3-5_all.deb ... 2021-03-09T02:46:28.322 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-logutils (0.3.3-5) ... 2021-03-09T02:46:28.326 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplegeneric. 2021-03-09T02:46:28.338 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../034-python3-simplegeneric_0.8.1-1_all.deb ... 2021-03-09T02:46:28.378 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-03-09T02:46:28.641 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-markupsafe. 2021-03-09T02:46:28.646 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-singledispatch. 2021-03-09T02:46:28.653 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../032-python3-markupsafe_1.0-1build1_amd64.deb ... 2021-03-09T02:46:28.658 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../035-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-03-09T02:46:28.691 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2021-03-09T02:46:28.713 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-03-09T02:46:28.960 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-mako. 2021-03-09T02:46:28.971 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../033-python3-mako_1.0.7+ds1-1_all.deb ... 2021-03-09T02:46:29.003 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2021-03-09T02:46:29.032 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webob. 2021-03-09T02:46:29.043 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../036-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-03-09T02:46:29.083 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-03-09T02:46:29.264 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-simplegeneric. 2021-03-09T02:46:29.275 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../034-python3-simplegeneric_0.8.1-1_all.deb ... 2021-03-09T02:46:29.315 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2021-03-09T02:46:29.393 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-bs4. 2021-03-09T02:46:29.404 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../037-python3-bs4_4.6.0-1_all.deb ... 2021-03-09T02:46:29.447 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-03-09T02:46:29.611 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-singledispatch. 2021-03-09T02:46:29.620 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../035-python3-singledispatch_3.4.0.3-2_all.deb ... 2021-03-09T02:46:29.673 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2021-03-09T02:46:29.763 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-waitress. 2021-03-09T02:46:29.776 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../038-python3-waitress_1.0.1-1_all.deb ... 2021-03-09T02:46:29.813 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-03-09T02:46:29.996 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webob. 2021-03-09T02:46:30.010 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../036-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2021-03-09T02:46:30.043 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2021-03-09T02:46:30.132 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tempita. 2021-03-09T02:46:30.145 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-2_all.deb ... 2021-03-09T02:46:30.182 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-03-09T02:46:30.337 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-bs4. 2021-03-09T02:46:30.350 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../037-python3-bs4_4.6.0-1_all.deb ... 2021-03-09T02:46:30.387 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2021-03-09T02:46:30.477 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-paste. 2021-03-09T02:46:30.490 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../040-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-03-09T02:46:30.520 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-03-09T02:46:30.698 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-waitress. 2021-03-09T02:46:30.709 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../038-python3-waitress_1.0.1-1_all.deb ... 2021-03-09T02:46:30.767 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-waitress (1.0.1-1) ... 2021-03-09T02:46:30.838 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-03-09T02:46:30.850 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-03-09T02:46:30.888 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-03-09T02:46:31.092 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tempita. 2021-03-09T02:46:31.104 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../039-python3-tempita_0.5.2-2_all.deb ... 2021-03-09T02:46:31.142 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tempita (0.5.2-2) ... 2021-03-09T02:46:31.216 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastedeploy. 2021-03-09T02:46:31.229 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../042-python3-pastedeploy_1.5.2-4_all.deb ... 2021-03-09T02:46:31.266 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-03-09T02:46:31.403 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-paste. 2021-03-09T02:46:31.417 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../040-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2021-03-09T02:46:31.444 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-03-09T02:46:31.552 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webtest. 2021-03-09T02:46:31.564 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../043-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-03-09T02:46:31.602 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-03-09T02:46:31.747 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2021-03-09T02:46:31.758 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../041-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2021-03-09T02:46:31.797 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2021-03-09T02:46:31.896 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pecan. 2021-03-09T02:46:31.907 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../044-python3-pecan_1.2.1-2_all.deb ... 2021-03-09T02:46:31.946 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-03-09T02:46:32.124 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pastedeploy. 2021-03-09T02:46:32.136 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../042-python3-pastedeploy_1.5.2-4_all.deb ... 2021-03-09T02:46:32.166 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2021-03-09T02:46:32.259 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-werkzeug. 2021-03-09T02:46:32.271 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../045-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-03-09T02:46:32.309 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-03-09T02:46:32.461 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webtest. 2021-03-09T02:46:32.473 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../043-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2021-03-09T02:46:32.510 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2021-03-09T02:46:32.713 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr. 2021-03-09T02:46:32.725 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../046-ceph-mgr_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:32.763 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:32.830 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pecan. 2021-03-09T02:46:32.842 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../044-python3-pecan_1.2.1-2_all.deb ... 2021-03-09T02:46:32.880 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pecan (1.2.1-2) ... 2021-03-09T02:46:33.232 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mon. 2021-03-09T02:46:33.246 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../047-ceph-mon_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:33.251 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-werkzeug. 2021-03-09T02:46:33.262 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../045-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.1_all.deb ... 2021-03-09T02:46:33.283 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mon (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:33.300 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-03-09T02:46:33.663 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr. 2021-03-09T02:46:33.674 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../046-ceph-mgr_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:33.713 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:34.183 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mon. 2021-03-09T02:46:34.195 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../047-ceph-mon_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:34.233 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mon (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:34.261 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-osd. 2021-03-09T02:46:34.272 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../048-ceph-osd_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:34.311 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-osd (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:35.837 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-osd. 2021-03-09T02:46:35.848 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../048-ceph-osd_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:35.886 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-osd (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:36.366 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph. 2021-03-09T02:46:36.378 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../049-ceph_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:36.416 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:36.735 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-fuse. 2021-03-09T02:46:36.747 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../050-ceph-fuse_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:36.785 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-fuse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:37.184 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mds. 2021-03-09T02:46:37.195 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../051-ceph-mds_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:37.236 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mds (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:37.753 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package cephadm. 2021-03-09T02:46:37.765 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../052-cephadm_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:37.795 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking cephadm (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:37.883 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph. 2021-03-09T02:46:37.896 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../049-ceph_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:37.925 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:38.115 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jinja2. 2021-03-09T02:46:38.126 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../053-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-03-09T02:46:38.165 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-03-09T02:46:38.245 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-fuse. 2021-03-09T02:46:38.257 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../050-ceph-fuse_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:38.294 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-fuse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:38.484 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-03-09T02:46:38.496 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../054-ceph-mgr-cephadm_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:38.535 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-cephadm (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:38.747 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mds. 2021-03-09T02:46:38.759 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../051-ceph-mds_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:38.797 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mds (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:38.887 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-repoze.lru. 2021-03-09T02:46:38.899 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../055-python3-repoze.lru_0.7-1_all.deb ... 2021-03-09T02:46:38.937 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-03-09T02:46:39.240 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-routes. 2021-03-09T02:46:39.252 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../056-python3-routes_2.4.1-1_all.deb ... 2021-03-09T02:46:39.290 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-03-09T02:46:39.310 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package cephadm. 2021-03-09T02:46:39.320 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../052-cephadm_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:39.358 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking cephadm (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:39.601 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-03-09T02:46:39.613 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:39.627 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jinja2. 2021-03-09T02:46:39.640 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../053-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2021-03-09T02:46:39.651 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-dashboard (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:39.694 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-03-09T02:46:39.972 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2021-03-09T02:46:39.982 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../054-ceph-mgr-cephadm_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:40.021 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-cephadm (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:40.325 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-repoze.lru. 2021-03-09T02:46:40.336 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../055-python3-repoze.lru_0.7-1_all.deb ... 2021-03-09T02:46:40.374 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2021-03-09T02:46:40.494 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-numpy. 2021-03-09T02:46:40.508 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../058-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-03-09T02:46:40.543 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-03-09T02:46:40.677 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-routes. 2021-03-09T02:46:40.689 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../056-python3-routes_2.4.1-1_all.deb ... 2021-03-09T02:46:40.747 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-routes (2.4.1-1) ... 2021-03-09T02:46:41.072 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2021-03-09T02:46:41.085 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../057-ceph-mgr-dashboard_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:41.121 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-dashboard (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:41.156 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-decorator. 2021-03-09T02:46:41.169 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../059-python3-decorator_4.1.2-1_all.deb ... 2021-03-09T02:46:41.206 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-03-09T02:46:41.535 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-scipy. 2021-03-09T02:46:41.546 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../060-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-03-09T02:46:41.583 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-03-09T02:46:41.979 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-numpy. 2021-03-09T02:46:41.994 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../058-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2021-03-09T02:46:42.028 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2021-03-09T02:46:42.574 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-decorator. 2021-03-09T02:46:42.590 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../059-python3-decorator_4.1.2-1_all.deb ... 2021-03-09T02:46:42.632 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-decorator (4.1.2-1) ... 2021-03-09T02:46:42.700 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-03-09T02:46:42.712 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-03-09T02:46:42.750 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-03-09T02:46:42.919 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-scipy. 2021-03-09T02:46:42.932 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../060-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2021-03-09T02:46:42.994 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2021-03-09T02:46:43.206 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-joblib. 2021-03-09T02:46:43.211 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../062-python3-joblib_0.11-1_all.deb ... 2021-03-09T02:46:43.253 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-joblib (0.11-1) ... 2021-03-09T02:46:43.598 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sklearn. 2021-03-09T02:46:43.611 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../063-python3-sklearn_0.19.1-3_all.deb ... 2021-03-09T02:46:43.648 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-03-09T02:46:44.152 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sklearn-lib. 2021-03-09T02:46:44.165 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../061-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2021-03-09T02:46:44.200 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-03-09T02:46:44.210 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2021-03-09T02:46:44.212 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:44.250 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-diskprediction-local (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:44.646 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-joblib. 2021-03-09T02:46:44.660 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../062-python3-joblib_0.11-1_all.deb ... 2021-03-09T02:46:44.696 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-joblib (0.11-1) ... 2021-03-09T02:46:45.133 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sklearn. 2021-03-09T02:46:45.145 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../063-python3-sklearn_0.19.1-3_all.deb ... 2021-03-09T02:46:45.217 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2021-03-09T02:46:45.420 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-monotonic. 2021-03-09T02:46:45.433 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../065-python3-monotonic_1.1-2_all.deb ... 2021-03-09T02:46:45.470 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-03-09T02:46:45.723 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2021-03-09T02:46:45.737 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../064-ceph-mgr-diskprediction-local_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:45.781 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-diskprediction-local (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:45.782 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-fasteners. 2021-03-09T02:46:45.793 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../066-python3-fasteners_0.12.0-3_all.deb ... 2021-03-09T02:46:45.831 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-03-09T02:46:46.109 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-httplib2. 2021-03-09T02:46:46.128 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../067-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2021-03-09T02:46:46.151 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-03-09T02:46:46.429 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1. 2021-03-09T02:46:46.442 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../068-python3-pyasn1_0.4.2-3_all.deb ... 2021-03-09T02:46:46.504 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-03-09T02:46:46.832 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-03-09T02:46:46.844 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../069-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-03-09T02:46:46.882 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-03-09T02:46:46.884 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-monotonic. 2021-03-09T02:46:46.897 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../065-python3-monotonic_1.1-2_all.deb ... 2021-03-09T02:46:46.934 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-monotonic (1.1-2) ... 2021-03-09T02:46:47.168 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-rsa. 2021-03-09T02:46:47.182 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../070-python3-rsa_3.4.2-1_all.deb ... 2021-03-09T02:46:47.218 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-03-09T02:46:47.220 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-fasteners. 2021-03-09T02:46:47.233 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../066-python3-fasteners_0.12.0-3_all.deb ... 2021-03-09T02:46:47.270 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2021-03-09T02:46:47.530 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-03-09T02:46:47.543 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../071-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-03-09T02:46:47.588 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-03-09T02:46:47.600 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-httplib2. 2021-03-09T02:46:47.612 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../067-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2021-03-09T02:46:47.650 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-03-09T02:46:47.936 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyasn1. 2021-03-09T02:46:47.947 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../068-python3-pyasn1_0.4.2-3_all.deb ... 2021-03-09T02:46:47.985 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2021-03-09T02:46:48.035 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-oauth2client. 2021-03-09T02:46:48.048 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../072-python3-oauth2client_4.1.2-3_all.deb ... 2021-03-09T02:46:48.084 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-03-09T02:46:48.322 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyasn1-modules. 2021-03-09T02:46:48.334 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../069-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2021-03-09T02:46:48.372 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2021-03-09T02:46:48.404 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-setuptools. 2021-03-09T02:46:48.416 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../073-python3-setuptools_39.0.1-2_all.deb ... 2021-03-09T02:46:48.453 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-03-09T02:46:48.708 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-rsa. 2021-03-09T02:46:48.721 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../070-python3-rsa_3.4.2-1_all.deb ... 2021-03-09T02:46:48.758 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-rsa (3.4.2-1) ... 2021-03-09T02:46:48.841 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-websocket. 2021-03-09T02:46:48.853 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../074-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-03-09T02:46:48.891 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-03-09T02:46:49.036 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-sqlalchemy. 2021-03-09T02:46:49.050 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../071-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2021-03-09T02:46:49.086 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-03-09T02:46:49.227 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-kubernetes. 2021-03-09T02:46:49.240 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../075-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-03-09T02:46:49.285 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-03-09T02:46:49.524 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-oauth2client. 2021-03-09T02:46:49.537 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../072-python3-oauth2client_4.1.2-3_all.deb ... 2021-03-09T02:46:49.607 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2021-03-09T02:46:49.698 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-03-09T02:46:49.710 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:49.748 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-k8sevents (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:50.176 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-setuptools. 2021-03-09T02:46:50.188 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../073-python3-setuptools_39.0.1-2_all.deb ... 2021-03-09T02:46:50.226 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2021-03-09T02:46:50.333 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-json-pointer. 2021-03-09T02:46:50.345 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../077-python3-json-pointer_1.10-1_all.deb ... 2021-03-09T02:46:50.383 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-03-09T02:46:50.613 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-websocket. 2021-03-09T02:46:50.625 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../074-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2021-03-09T02:46:50.663 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2021-03-09T02:46:50.686 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-jsonpatch. 2021-03-09T02:46:50.700 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../078-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-03-09T02:46:50.740 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-03-09T02:46:50.994 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-kubernetes. 2021-03-09T02:46:51.002 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../075-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2021-03-09T02:46:51.040 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-03-09T02:46:51.041 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2021-03-09T02:46:51.052 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../079-ceph-mgr-rook_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:51.090 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-mgr-rook (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:51.418 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libonig4:amd64. 2021-03-09T02:46:51.430 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2021-03-09T02:46:51.470 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2021-03-09T02:46:51.483 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../076-ceph-mgr-k8sevents_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:51.520 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-k8sevents (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:51.545 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-03-09T02:46:51.806 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-json-pointer. 2021-03-09T02:46:51.819 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../077-python3-json-pointer_1.10-1_all.deb ... 2021-03-09T02:46:51.855 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-json-pointer (1.10-1) ... 2021-03-09T02:46:51.881 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libjq1:amd64. 2021-03-09T02:46:51.895 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2021-03-09T02:46:51.939 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-03-09T02:46:52.209 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-jsonpatch. 2021-03-09T02:46:52.210 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package jq. 2021-03-09T02:46:52.221 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../078-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2021-03-09T02:46:52.222 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2021-03-09T02:46:52.261 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-03-09T02:46:52.267 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-03-09T02:46:52.520 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package socat. 2021-03-09T02:46:52.533 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-03-09T02:46:52.545 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-mgr-rook. 2021-03-09T02:46:52.558 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../079-ceph-mgr-rook_16.1.0-627-g9e448db9-1bionic_all.deb ... 2021-03-09T02:46:52.578 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-03-09T02:46:52.603 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-mgr-rook (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:52.881 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libonig4:amd64. 2021-03-09T02:46:52.895 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2021-03-09T02:46:52.915 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-03-09T02:46:52.928 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-03-09T02:46:52.956 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-03-09T02:46:53.025 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2021-03-09T02:46:53.218 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package xmlstarlet. 2021-03-09T02:46:53.229 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2021-03-09T02:46:53.268 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-03-09T02:46:53.377 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libjq1:amd64. 2021-03-09T02:46:53.391 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2021-03-09T02:46:53.436 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2021-03-09T02:46:53.589 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package ceph-test. 2021-03-09T02:46:53.601 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../086-ceph-test_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:53.638 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking ceph-test (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:53.705 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package jq. 2021-03-09T02:46:53.717 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2021-03-09T02:46:53.755 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking jq (1.5+dfsg-2) ... 2021-03-09T02:46:54.016 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package socat. 2021-03-09T02:46:54.029 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2021-03-09T02:46:54.066 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2021-03-09T02:46:54.419 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libxslt1.1:amd64. 2021-03-09T02:46:54.432 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2021-03-09T02:46:54.461 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-03-09T02:46:54.722 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package xmlstarlet. 2021-03-09T02:46:54.734 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2021-03-09T02:46:54.772 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2021-03-09T02:46:55.110 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package ceph-test. 2021-03-09T02:46:55.122 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../086-ceph-test_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:55.159 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking ceph-test (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:57.017 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package libcephfs-dev. 2021-03-09T02:46:57.030 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../087-libcephfs-dev_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:57.083 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking libcephfs-dev (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:57.361 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package nvme-cli. 2021-03-09T02:46:57.373 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-03-09T02:46:57.411 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-03-09T02:46:57.766 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-attr. 2021-03-09T02:46:57.780 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2021-03-09T02:46:57.824 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-attr (17.4.0-2) ... 2021-03-09T02:46:58.136 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-funcsigs. 2021-03-09T02:46:58.147 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2021-03-09T02:46:58.185 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-03-09T02:46:58.471 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package libcephfs-dev. 2021-03-09T02:46:58.480 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-pluggy. 2021-03-09T02:46:58.484 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../087-libcephfs-dev_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:46:58.494 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2021-03-09T02:46:58.538 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-03-09T02:46:58.545 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking libcephfs-dev (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:46:58.811 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-py. 2021-03-09T02:46:58.822 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../092-python-py_1.5.2-1_all.deb ... 2021-03-09T02:46:58.832 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package nvme-cli. 2021-03-09T02:46:58.843 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.1_amd64.deb ... 2021-03-09T02:46:58.858 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-py (1.5.2-1) ... 2021-03-09T02:46:58.882 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking nvme-cli (1.5-1ubuntu1.1) ... 2021-03-09T02:46:59.161 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python-pytest. 2021-03-09T02:46:59.174 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2021-03-09T02:46:59.211 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-03-09T02:46:59.220 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-attr. 2021-03-09T02:46:59.234 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2021-03-09T02:46:59.278 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-attr (17.4.0-2) ... 2021-03-09T02:46:59.506 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-attr. 2021-03-09T02:46:59.519 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2021-03-09T02:46:59.557 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-03-09T02:46:59.606 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-funcsigs. 2021-03-09T02:46:59.620 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2021-03-09T02:46:59.664 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2021-03-09T02:46:59.875 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-cycler. 2021-03-09T02:46:59.888 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2021-03-09T02:46:59.926 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-03-09T02:46:59.967 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-pluggy. 2021-03-09T02:46:59.978 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2021-03-09T02:47:00.017 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-pluggy (0.6.0-1) ... 2021-03-09T02:47:00.204 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-webencodings. 2021-03-09T02:47:00.217 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2021-03-09T02:47:00.261 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-03-09T02:47:00.319 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-py. 2021-03-09T02:47:00.332 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../092-python-py_1.5.2-1_all.deb ... 2021-03-09T02:47:00.369 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-py (1.5.2-1) ... 2021-03-09T02:47:00.591 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-html5lib. 2021-03-09T02:47:00.603 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2021-03-09T02:47:00.633 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-03-09T02:47:00.706 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python-pytest. 2021-03-09T02:47:00.718 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2021-03-09T02:47:00.747 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python-pytest (3.3.2-2) ... 2021-03-09T02:47:00.934 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-03-09T02:47:00.948 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-03-09T02:47:00.992 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-03-09T02:47:01.059 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-attr. 2021-03-09T02:47:01.072 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2021-03-09T02:47:01.109 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-attr (17.4.0-2) ... 2021-03-09T02:47:01.378 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-cycler. 2021-03-09T02:47:01.388 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyparsing. 2021-03-09T02:47:01.390 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2021-03-09T02:47:01.401 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-03-09T02:47:01.428 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-cycler (0.10.0-1) ... 2021-03-09T02:47:01.437 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-03-09T02:47:01.765 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tz. 2021-03-09T02:47:01.777 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2021-03-09T02:47:01.815 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tz (2018.3-2) ... 2021-03-09T02:47:01.856 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-webencodings. 2021-03-09T02:47:01.870 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2021-03-09T02:47:01.914 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-webencodings (0.5-2) ... 2021-03-09T02:47:02.118 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-matplotlib. 2021-03-09T02:47:02.129 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-03-09T02:47:02.168 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-03-09T02:47:02.200 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-html5lib. 2021-03-09T02:47:02.214 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2021-03-09T02:47:02.250 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2021-03-09T02:47:02.587 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-lxml:amd64. 2021-03-09T02:47:02.599 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.3_amd64.deb ... 2021-03-09T02:47:02.636 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-03-09T02:47:02.791 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-olefile. 2021-03-09T02:47:02.802 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2021-03-09T02:47:02.832 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-03-09T02:47:03.091 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyparsing. 2021-03-09T02:47:03.102 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2021-03-09T02:47:03.140 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2021-03-09T02:47:03.152 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pastescript. 2021-03-09T02:47:03.164 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2021-03-09T02:47:03.201 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-03-09T02:47:03.442 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tz. 2021-03-09T02:47:03.456 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2021-03-09T02:47:03.484 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tz (2018.3-2) ... 2021-03-09T02:47:03.580 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pil:amd64. 2021-03-09T02:47:03.593 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.4_amd64.deb ... 2021-03-09T02:47:03.638 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-03-09T02:47:03.787 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-matplotlib. 2021-03-09T02:47:03.800 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2021-03-09T02:47:03.837 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2021-03-09T02:47:04.000 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pluggy. 2021-03-09T02:47:04.014 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2021-03-09T02:47:04.058 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-03-09T02:47:04.394 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-py. 2021-03-09T02:47:04.407 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../106-python3-py_1.5.2-1_all.deb ... 2021-03-09T02:47:04.444 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-py (1.5.2-1) ... 2021-03-09T02:47:04.517 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-olefile. 2021-03-09T02:47:04.529 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2021-03-09T02:47:04.567 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-olefile (0.45.1-1) ... 2021-03-09T02:47:04.756 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pyinotify. 2021-03-09T02:47:04.770 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2021-03-09T02:47:04.814 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-03-09T02:47:04.878 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pastescript. 2021-03-09T02:47:04.891 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2021-03-09T02:47:04.928 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2021-03-09T02:47:05.109 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-pytest. 2021-03-09T02:47:05.121 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2021-03-09T02:47:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-03-09T02:47:05.315 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pil:amd64. 2021-03-09T02:47:05.328 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.4_amd64.deb ... 2021-03-09T02:47:05.365 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-03-09T02:47:05.520 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-simplejson. 2021-03-09T02:47:05.533 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2021-03-09T02:47:05.570 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-03-09T02:47:05.726 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pluggy. 2021-03-09T02:47:05.738 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2021-03-09T02:47:05.776 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2021-03-09T02:47:05.890 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package python3-tk:amd64. 2021-03-09T02:47:05.903 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-03-09T02:47:05.932 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-03-09T02:47:06.045 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-py. 2021-03-09T02:47:06.058 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../106-python3-py_1.5.2-1_all.deb ... 2021-03-09T02:47:06.087 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-py (1.5.2-1) ... 2021-03-09T02:47:06.440 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pyinotify. 2021-03-09T02:47:06.453 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2021-03-09T02:47:06.491 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2021-03-09T02:47:06.511 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package radosgw. 2021-03-09T02:47:06.522 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../111-radosgw_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:47:06.560 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking radosgw (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:06.801 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-pytest. 2021-03-09T02:47:06.813 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2021-03-09T02:47:06.851 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-pytest (3.3.2-2) ... 2021-03-09T02:47:07.196 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-simplejson. 2021-03-09T02:47:07.207 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2021-03-09T02:47:07.246 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2021-03-09T02:47:07.581 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package rbd-fuse. 2021-03-09T02:47:07.592 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../112-rbd-fuse_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:47:07.616 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package python3-tk:amd64. 2021-03-09T02:47:07.629 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2021-03-09T02:47:07.630 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking rbd-fuse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:07.666 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2021-03-09T02:47:07.933 INFO:teuthology.orchestra.run.smithi144.stdout:Selecting previously unselected package rbd-mirror. 2021-03-09T02:47:07.948 INFO:teuthology.orchestra.run.smithi144.stdout:Preparing to unpack .../113-rbd-mirror_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:47:08.006 INFO:teuthology.orchestra.run.smithi144.stdout:Unpacking rbd-mirror (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:08.111 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package radosgw. 2021-03-09T02:47:08.122 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../111-radosgw_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:47:08.162 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking radosgw (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:08.606 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-03-09T02:47:08.906 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-03-09T02:47:09.114 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package rbd-fuse. 2021-03-09T02:47:09.125 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../112-rbd-fuse_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:47:09.172 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking rbd-fuse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:09.299 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-03-09T02:47:09.416 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-03-09T02:47:09.517 INFO:teuthology.orchestra.run.smithi092.stdout:Selecting previously unselected package rbd-mirror. 2021-03-09T02:47:09.529 INFO:teuthology.orchestra.run.smithi092.stdout:Preparing to unpack .../113-rbd-mirror_16.1.0-627-g9e448db9-1bionic_amd64.deb ... 2021-03-09T02:47:09.591 INFO:teuthology.orchestra.run.smithi092.stdout:Unpacking rbd-mirror (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:10.002 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-03-09T02:47:10.209 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-logutils (0.3.3-5) ... 2021-03-09T02:47:10.387 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-03-09T02:47:10.522 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.1) ... 2021-03-09T02:47:10.679 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webencodings (0.5-2) ... 2021-03-09T02:47:10.881 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2021-03-09T02:47:10.989 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-attr (17.4.0-2) ... 2021-03-09T02:47:11.007 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2021-03-09T02:47:11.182 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-03-09T02:47:11.300 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-py (1.5.2-1) ... 2021-03-09T02:47:11.618 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.4) ... 2021-03-09T02:47:11.626 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-03-09T02:47:11.977 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-03-09T02:47:12.011 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2021-03-09T02:47:12.279 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-03-09T02:47:12.287 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webencodings (0.5-2) ... 2021-03-09T02:47:12.566 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-argparse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:12.580 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-attr (17.4.0-2) ... 2021-03-09T02:47:12.773 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2021-03-09T02:47:12.851 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-03-09T02:47:12.924 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-py (1.5.2-1) ... 2021-03-09T02:47:13.151 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-03-09T02:47:13.250 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-waitress (1.0.1-1) ... 2021-03-09T02:47:13.277 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-03-09T02:47:13.646 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2021-03-09T02:47:13.978 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-olefile (0.45.1-1) ... 2021-03-09T02:47:14.205 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-03-09T02:47:14.271 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-ceph-argparse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:14.523 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-03-09T02:47:14.564 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tempita (0.5.2-2) ... 2021-03-09T02:47:14.816 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-py (1.5.2-1) ... 2021-03-09T02:47:14.865 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2021-03-09T02:47:14.991 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2021-03-09T02:47:15.025 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-03-09T02:47:15.360 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-03-09T02:47:15.492 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-03-09T02:47:15.728 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-03-09T02:47:15.854 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-03-09T02:47:15.935 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2021-03-09T02:47:16.055 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-03-09T02:47:16.253 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2021-03-09T02:47:16.349 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-03-09T02:47:16.537 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-py (1.5.2-1) ... 2021-03-09T02:47:16.683 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-03-09T02:47:16.756 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-simplejson (3.13.2-1) ... 2021-03-09T02:47:16.967 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-03-09T02:47:17.090 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-json-pointer (1.10-1) ... 2021-03-09T02:47:17.180 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2021-03-09T02:47:17.335 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-03-09T02:47:17.425 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2021-03-09T02:47:17.551 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-pluggy (0.6.0-1) ... 2021-03-09T02:47:17.678 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-03-09T02:47:17.752 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2021-03-09T02:47:17.979 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up cephadm (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:18.053 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2021-03-09T02:47:18.379 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cycler (0.10.0-1) ... 2021-03-09T02:47:18.655 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2021-03-09T02:47:18.671 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user cephadm....done 2021-03-09T02:47:18.854 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-03-09T02:47:19.023 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-bs4 (4.6.0-1) ... 2021-03-09T02:47:19.145 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-03-09T02:47:19.271 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-03-09T02:47:19.349 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pluggy (0.6.0-1) ... 2021-03-09T02:47:19.447 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-03-09T02:47:19.667 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up cephadm (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:19.798 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-03-09T02:47:20.099 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-03-09T02:47:20.225 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-03-09T02:47:20.376 INFO:teuthology.orchestra.run.smithi092.stdout:Adding system user cephadm....done 2021-03-09T02:47:20.657 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2021-03-09T02:47:20.827 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-ceph-common (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:21.136 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-monotonic (1.1-2) ... 2021-03-09T02:47:21.158 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2021-03-09T02:47:21.284 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up nvme-cli (1.5-1ubuntu1.1) ... 2021-03-09T02:47:21.427 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2021-03-09T02:47:21.429 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-03-09T02:47:21.555 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-03-09T02:47:21.786 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-prettytable (0.7.2-3) ... 2021-03-09T02:47:21.898 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-03-09T02:47:22.103 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2021-03-09T02:47:22.213 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2021-03-09T02:47:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-03-09T02:47:22.339 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-03-09T02:47:22.371 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-03-09T02:47:22.523 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-ceph-common (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:22.602 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-03-09T02:47:22.841 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-monotonic (1.1-2) ... 2021-03-09T02:47:22.902 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-03-09T02:47:23.125 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2021-03-09T02:47:23.203 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-03-09T02:47:23.251 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2021-03-09T02:47:23.530 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-attr (17.4.0-2) ... 2021-03-09T02:47:23.594 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-html5lib (0.999999999-1) ... 2021-03-09T02:47:23.839 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-03-09T02:47:23.962 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2021-03-09T02:47:24.049 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libjaeger (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:24.051 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2021-03-09T02:47:24.075 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2021-03-09T02:47:24.174 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-03-09T02:47:24.314 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-repoze.lru (0.7-1) ... 2021-03-09T02:47:24.568 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-03-09T02:47:24.615 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2021-03-09T02:47:24.682 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-03-09T02:47:24.908 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2021-03-09T02:47:24.944 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liboath0 (2.6.1-1) ... 2021-03-09T02:47:25.120 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-03-09T02:47:25.247 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-03-09T02:47:25.267 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-attr (17.4.0-2) ... 2021-03-09T02:47:25.498 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-03-09T02:47:25.568 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-funcsigs (1.0.2-4) ... 2021-03-09T02:47:25.607 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-03-09T02:47:25.769 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libjaeger (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:25.895 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2021-03-09T02:47:25.959 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-03-09T02:47:26.084 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-03-09T02:47:26.288 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2021-03-09T02:47:26.377 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-tz (2018.3-2) ... 2021-03-09T02:47:26.387 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2021-03-09T02:47:26.632 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liboath0 (2.6.1-1) ... 2021-03-09T02:47:26.678 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-03-09T02:47:26.800 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2021-03-09T02:47:26.926 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2021-03-09T02:47:26.988 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-03-09T02:47:27.202 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2021-03-09T02:47:27.305 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-joblib (0.11-1) ... 2021-03-09T02:47:27.328 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-setuptools (39.0.1-2) ... 2021-03-09T02:47:27.657 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-routes (2.4.1-1) ... 2021-03-09T02:47:27.713 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2021-03-09T02:47:27.814 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-decorator (4.1.2-1) ... 2021-03-09T02:47:27.935 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-03-09T02:47:28.090 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-tz (2018.3-2) ... 2021-03-09T02:47:28.217 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-03-09T02:47:28.399 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2021-03-09T02:47:28.543 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-03-09T02:47:28.683 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2021-03-09T02:47:28.903 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-03-09T02:47:28.993 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-joblib (0.11-1) ... 2021-03-09T02:47:29.204 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up jq (1.5+dfsg-2) ... 2021-03-09T02:47:29.330 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-03-09T02:47:29.361 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-routes (2.4.1-1) ... 2021-03-09T02:47:29.704 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2021-03-09T02:47:29.988 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-dateutil (2.6.1-1) ... 2021-03-09T02:47:30.281 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2021-03-09T02:47:30.485 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-03-09T02:47:30.633 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-fasteners (0.12.0-3) ... 2021-03-09T02:47:30.811 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-03-09T02:47:30.934 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up jq (1.5+dfsg-2) ... 2021-03-09T02:47:31.060 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2021-03-09T02:47:31.171 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-03-09T02:47:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-03-09T02:47:31.665 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python-pytest (3.3.2-2) ... 2021-03-09T02:47:31.900 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-03-09T02:47:32.000 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-03-09T02:47:32.126 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librados2 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:32.231 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.3) ... 2021-03-09T02:47:32.277 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-03-09T02:47:32.549 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2021-03-09T02:47:32.595 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs2 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:32.721 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libcephfs-dev (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:32.847 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rados (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:32.875 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2021-03-09T02:47:32.972 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-03-09T02:47:33.001 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pytest (3.3.2-2) ... 2021-03-09T02:47:33.249 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-03-09T02:47:33.369 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python-pytest (3.3.2-2) ... 2021-03-09T02:47:33.550 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-03-09T02:47:33.646 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2021-03-09T02:47:33.755 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up xmlstarlet (1.6.1-2) ... 2021-03-09T02:47:33.889 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librados2 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:33.952 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-03-09T02:47:34.015 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rsa (3.4.2-1) ... 2021-03-09T02:47:34.333 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libcephfs2 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:34.459 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libcephfs-dev (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:34.584 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rados (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:34.727 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2021-03-09T02:47:34.797 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-03-09T02:47:35.030 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2021-03-09T02:47:35.315 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2021-03-09T02:47:35.635 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-03-09T02:47:35.726 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2021-03-09T02:47:35.969 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up libradosstriper1 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:36.096 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-cephfs (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:36.396 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librgw2 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:36.522 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-fuse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:36.571 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-sklearn (0.19.1-3) ... 2021-03-09T02:47:36.740 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-03-09T02:47:36.740 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-03-09T02:47:36.999 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up librbd1 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:37.099 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-03-09T02:47:37.180 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-03-09T02:47:37.238 INFO:teuthology.orchestra.run.smithi144.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-03-09T02:47:37.400 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2021-03-09T02:47:37.520 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-03-09T02:47:37.676 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up libradosstriper1 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:37.777 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-cephfs (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:38.053 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librgw2 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:38.172 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-modules-core (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:38.179 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-fuse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:38.298 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-03-09T02:47:38.430 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-03-09T02:47:38.431 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2021-03-09T02:47:38.599 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rgw (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:38.700 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up rbd-fuse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:38.714 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up librbd1 (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:38.801 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up python3-rbd (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:38.840 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pecan (1.2.1-2) ... 2021-03-09T02:47:38.927 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-common (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:38.930 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2021-03-09T02:47:38.971 INFO:teuthology.orchestra.run.smithi092.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2021-03-09T02:47:39.260 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2021-03-09T02:47:39.461 INFO:teuthology.orchestra.run.smithi144.stdout:Adding group ceph....done 2021-03-09T02:47:39.830 INFO:teuthology.orchestra.run.smithi144.stdout:Adding system user ceph....done 2021-03-09T02:47:39.929 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-modules-core (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:40.022 INFO:teuthology.orchestra.run.smithi144.stdout:Setting system user ceph properties....done 2021-03-09T02:47:40.055 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-pastescript (2.0.2-2) ... 2021-03-09T02:47:40.173 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-03-09T02:47:40.373 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rgw (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:40.387 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-03-09T02:47:40.524 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up rbd-fuse (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:40.633 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up python3-rbd (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:40.678 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up radosgw (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:40.734 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-common (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:40.941 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-03-09T02:47:40.941 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-03-09T02:47:41.294 INFO:teuthology.orchestra.run.smithi092.stdout:Adding group ceph....done 2021-03-09T02:47:41.331 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up rbd-mirror (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:41.542 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-03-09T02:47:41.542 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-03-09T02:47:41.696 INFO:teuthology.orchestra.run.smithi092.stdout:Adding system user ceph....done 2021-03-09T02:47:41.815 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-test (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:41.889 INFO:teuthology.orchestra.run.smithi092.stdout:Setting system user ceph properties....done 2021-03-09T02:47:41.918 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-base (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:42.039 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2021-03-09T02:47:42.226 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-03-09T02:47:42.281 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2021-03-09T02:47:42.510 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:42.586 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up radosgw (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:42.728 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-03-09T02:47:42.728 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-03-09T02:47:42.870 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-03-09T02:47:42.870 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2021-03-09T02:47:43.002 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-diskprediction-local (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:43.145 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-osd (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:43.288 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up rbd-mirror (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:43.498 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-03-09T02:47:43.499 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target -> /lib/systemd/system/ceph-rbd-mirror.target. 2021-03-09T02:47:43.595 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-03-09T02:47:43.595 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-03-09T02:47:43.781 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-test (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:43.905 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-cephadm (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:43.907 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-base (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:44.048 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-rook (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:44.166 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mds (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:44.241 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2021-03-09T02:47:44.370 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-03-09T02:47:44.370 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-03-09T02:47:44.534 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:44.659 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mon (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:44.728 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-03-09T02:47:44.729 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2021-03-09T02:47:44.871 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-03-09T02:47:44.871 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-03-09T02:47:45.010 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-diskprediction-local (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:45.128 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-osd (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:45.160 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-k8sevents (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:45.286 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph-mgr-dashboard (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:45.429 INFO:teuthology.orchestra.run.smithi144.stdout:Setting up ceph (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:45.555 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for systemd (237-3ubuntu10.44) ... 2021-03-09T02:47:45.573 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-03-09T02:47:45.573 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2021-03-09T02:47:45.748 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-03-09T02:47:45.855 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-cephadm (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:46.006 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-rook (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:46.148 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mds (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:46.587 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-03-09T02:47:46.588 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2021-03-09T02:47:46.868 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mon (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:47.067 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-03-09T02:47:47.067 INFO:teuthology.orchestra.run.smithi092.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2021-03-09T02:47:47.361 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-k8sevents (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:47.486 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph-mgr-dashboard (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:47.638 INFO:teuthology.orchestra.run.smithi092.stdout:Setting up ceph (16.1.0-627-g9e448db9-1bionic) ... 2021-03-09T02:47:47.764 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for systemd (237-3ubuntu10.44) ... 2021-03-09T02:47:47.957 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2021-03-09T02:47:47.976 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-03-09T02:47:48.010 INFO:teuthology.orchestra.run.smithi144.stdout:ureadahead will be reprofiled on next reboot 2021-03-09T02:47:48.060 INFO:teuthology.orchestra.run.smithi144.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-03-09T02:47:48.521 INFO:teuthology.orchestra.run.smithi144.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-03-09T02:47:48.523 DEBUG:teuthology.parallel:result is None 2021-03-09T02:47:50.134 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2021-03-09T02:47:50.169 INFO:teuthology.orchestra.run.smithi092.stdout:ureadahead will be reprofiled on next reboot 2021-03-09T02:47:50.210 INFO:teuthology.orchestra.run.smithi092.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.4) ... 2021-03-09T02:47:50.687 INFO:teuthology.orchestra.run.smithi092.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2021-03-09T02:47:50.689 DEBUG:teuthology.parallel:result is None 2021-03-09T02:47:50.690 DEBUG:teuthology.orchestra.run.smithi092:> sudo lsb_release -is 2021-03-09T02:47:50.737 INFO:teuthology.orchestra.run.smithi092.stdout:Ubuntu 2021-03-09T02:47:50.742 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-09T02:47:50.743 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=9e448db9e385343da144613d840349182443aeee 2021-03-09T02:47:50.892 DEBUG:teuthology.orchestra.run.smithi092:> dpkg-query -W -f '${Version}' ceph 2021-03-09T02:47:50.917 INFO:teuthology.orchestra.run.smithi092.stdout:16.1.0-627-g9e448db9-1bionic 2021-03-09T02:47:50.918 INFO:teuthology.packaging:The installed version of ceph is 16.1.0-627-g9e448db9-1bionic 2021-03-09T02:47:50.918 INFO:teuthology.task.install:The correct ceph version 16.1.0-627-g9e448db9-1bionic is installed. 2021-03-09T02:47:50.919 DEBUG:teuthology.orchestra.run.smithi144:> sudo lsb_release -is 2021-03-09T02:47:50.977 INFO:teuthology.orchestra.run.smithi144.stdout:Ubuntu 2021-03-09T02:47:50.982 DEBUG:teuthology.misc:System to be installed: Ubuntu 2021-03-09T02:47:50.983 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=9e448db9e385343da144613d840349182443aeee 2021-03-09T02:47:51.136 DEBUG:teuthology.orchestra.run.smithi144:> dpkg-query -W -f '${Version}' ceph 2021-03-09T02:47:51.162 INFO:teuthology.orchestra.run.smithi144.stdout:16.1.0-627-g9e448db9-1bionic 2021-03-09T02:47:51.163 INFO:teuthology.packaging:The installed version of ceph is 16.1.0-627-g9e448db9-1bionic 2021-03-09T02:47:51.163 INFO:teuthology.task.install:The correct ceph version 16.1.0-627-g9e448db9-1bionic is installed. 2021-03-09T02:47:51.164 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-03-09T02:47:51.164 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:47:51.164 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-03-09T02:47:51.180 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:47:51.180 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-03-09T02:47:51.218 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-03-09T02:47:51.219 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:47:51.219 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/daemon-helper 2021-03-09T02:47:51.238 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-03-09T02:47:51.292 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:47:51.293 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2021-03-09T02:47:51.307 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-03-09T02:47:51.366 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-03-09T02:47:51.367 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:47:51.367 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/usr/bin/adjust-ulimits 2021-03-09T02:47:51.386 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-03-09T02:47:51.442 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:47:51.442 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2021-03-09T02:47:51.457 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-03-09T02:47:51.514 INFO:teuthology.run_tasks:Running task ceph... 2021-03-09T02:47:51.581 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2021-03-09T02:47:51.582 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 777 /var/log/ceph 2021-03-09T02:47:51.585 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /var/log/ceph 2021-03-09T02:47:51.600 INFO:tasks.ceph:Disabling ceph logrotate... 2021-03-09T02:47:51.600 DEBUG:teuthology.orchestra.run.smithi092:> sudo rm -f -- /etc/logrotate.d/ceph 2021-03-09T02:47:51.641 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/logrotate.d/ceph 2021-03-09T02:47:51.657 INFO:tasks.ceph:Creating extra log directories... 2021-03-09T02:47:51.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-03-09T02:47:51.702 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2021-03-09T02:47:51.738 INFO:tasks.ceph:Creating ceph cluster cluster1... 2021-03-09T02:47:51.738 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster1', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-03-09T02:47:51.738 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2021-03-08_21:10:20-rbd-wip-yuri5-testing-2021-03-08-1049-pacific-distro-basic-smithi/5947743', 'branch': 'wip-yuri5-testing-2021-03-08-1049-pacific', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '5947743', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'yuriw-2021-03-08_21:10:20-rbd-wip-yuri5-testing-2021-03-08-1049-pacific-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri5-testing-2021-03-08-1049-pacific'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '9e448db9e385343da144613d840349182443aeee'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'sha1': '9e448db9e385343da144613d840349182443aeee'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri5-testing-2021-03-08-1049-pacific', 'sha1': '9e448db9e385343da144613d840349182443aeee'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 80, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'sha1': '9e448db9e385343da144613d840349182443aeee', 'sleep_before_teardown': 0, 'suite': 'rbd', 'suite_branch': 'wip-yuri5-testing-2021-03-08-1049-pacific', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_9e448db9e385343da144613d840349182443aeee/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '9e448db9e385343da144613d840349182443aeee', 'targets': {'smithi092.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGM9Wq+tpjME44vX4r5HVAYDKQHOPpKMuKkIN+sN4bXOXrL38m4IlDpX7+u11H/YKmLYO9J8qq8LeIhaPVveNNtI00S6NRcL2gCFTKBc3cGAuajI1w7T5UcmQcjTDdaC6irQDmclSzc/LrhabDa/E8LWZa/J4pJf2j93rHf5ALtl6DxhOPtDMtksmxnIlvXWznZj0nTqZXZjx2V3DJ8edC0ENLQTQuG/GIUX4uAXK5T40vzv21g1EG4jAiUKZXB1t91+utaws6miPeVZ6W94mAkMdmqdlkcbccy4PWRXiz4+MNFNhUtNBxtSgeuFc8gQfrZXKWdIxyHpSoybONFFN/', 'smithi144.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQClEsHrqwB5WWdjBzHc5HfVxIvP9a9tTOK/FXO13yldAlBA6wEnj8ybBtfnvsWbhjdORYKBxUEiPAcNjeDd6CdsyLAIAI/Yo6n//ZuBnBEURpcBxLmeyQf8YLvKLJ8eUYJTdiFMUiimvqszh7S1Uzqi5crdAP9jWyZjL7Bo+7itY52pzDWhxFtARQfcV/FOUBQymt8GK7We3s1Dlc/vfHw0JTaxeUfwCBOx2L2GxeGAGLpnnJhV6ohsFhIb7ZnfMRAR2PV4osf84S67vArQj7H5H/XqQJaSVc5EUkPodGwiRxqplJCgtlpb205SiEMOBwB/XX5ppSSCp0RfBj91NwdX'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'sha1': '9e448db9e385343da144613d840349182443aeee'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '9e448db9e385343da144613d840349182443aeee'}}, {'ceph': {'cluster': 'cluster2'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_IMAGE_MODE': 'snapshot', 'MIRROR_POOL_MODE': 'image', 'RBD_IMAGE_FEATURES': 'layering,exclusive-lock', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology_branch': 'master', 'teuthology_sha1': '3082387bba74fcd24c9700593d10418152d53c97', 'timestamp': '2021-03-08_21:10:20', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.25973'} 2021-03-09T02:47:51.740 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2021-03-09T02:47:51.761 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/cluster1.data 2021-03-09T02:47:51.769 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2021-03-09T02:47:51.809 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2021-03-09T02:47:51.826 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:47:51.827 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/scratch_devs of=/dev/stdout 2021-03-09T02:47:51.870 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-03-09T02:47:51.871 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-03-09T02:47:51.919 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-03-09T02:47:51.920 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-09T02:47:51.920 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 341 Links: 1 2021-03-09T02:47:51.920 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-09T02:47:51.920 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2021-03-09 02:45:18.667718623 +0000 2021-03-09T02:47:51.921 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2021-03-09 02:45:18.587718623 +0000 2021-03-09T02:47:51.921 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2021-03-09 02:45:18.587718623 +0000 2021-03-09T02:47:51.921 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2021-03-09T02:47:51.929 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2021-03-09T02:47:51.929 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2021-03-09T02:47:51.929 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000399601 s, 1.3 MB/s 2021-03-09T02:47:51.937 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-03-09T02:47:51.986 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-03-09T02:47:51.986 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-09T02:47:51.987 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 348 Links: 1 2021-03-09T02:47:51.987 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-09T02:47:51.987 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2021-03-09 02:45:18.923718620 +0000 2021-03-09T02:47:51.987 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2021-03-09 02:45:18.815718621 +0000 2021-03-09T02:47:51.988 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2021-03-09 02:45:18.815718621 +0000 2021-03-09T02:47:51.988 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2021-03-09T02:47:51.995 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2021-03-09T02:47:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2021-03-09T02:47:51.996 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000288107 s, 1.8 MB/s 2021-03-09T02:47:52.000 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-03-09T02:47:52.050 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-03-09T02:47:52.051 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-09T02:47:52.051 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 354 Links: 1 2021-03-09T02:47:52.051 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-09T02:47:52.051 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2021-03-09 02:45:19.183718617 +0000 2021-03-09T02:47:52.052 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2021-03-09 02:45:19.079718618 +0000 2021-03-09T02:47:52.052 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2021-03-09 02:45:19.079718618 +0000 2021-03-09T02:47:52.052 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2021-03-09T02:47:52.059 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2021-03-09T02:47:52.060 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2021-03-09T02:47:52.060 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000354105 s, 1.4 MB/s 2021-03-09T02:47:52.064 DEBUG:teuthology.orchestra.run.smithi092:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-03-09T02:47:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-03-09T02:47:52.118 INFO:teuthology.orchestra.run.smithi092.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-09T02:47:52.119 INFO:teuthology.orchestra.run.smithi092.stdout:Device: 6h/6d Inode: 360 Links: 1 2021-03-09T02:47:52.119 INFO:teuthology.orchestra.run.smithi092.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-09T02:47:52.119 INFO:teuthology.orchestra.run.smithi092.stdout:Access: 2021-03-09 02:45:19.435718614 +0000 2021-03-09T02:47:52.119 INFO:teuthology.orchestra.run.smithi092.stdout:Modify: 2021-03-09 02:45:19.311718616 +0000 2021-03-09T02:47:52.119 INFO:teuthology.orchestra.run.smithi092.stdout:Change: 2021-03-09 02:45:19.311718616 +0000 2021-03-09T02:47:52.120 INFO:teuthology.orchestra.run.smithi092.stdout: Birth: - 2021-03-09T02:47:52.128 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records in 2021-03-09T02:47:52.128 INFO:teuthology.orchestra.run.smithi092.stderr:1+0 records out 2021-03-09T02:47:52.128 INFO:teuthology.orchestra.run.smithi092.stderr:512 bytes copied, 0.000409348 s, 1.3 MB/s 2021-03-09T02:47:52.133 INFO:tasks.ceph:osd dev map: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-09T02:47:52.133 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2021-03-09T02:47:52.133 INFO:tasks.ceph:Generating config... 2021-03-09T02:47:52.135 INFO:tasks.ceph:[client] debug rbd = 20 2021-03-09T02:47:52.135 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2021-03-09T02:47:52.135 INFO:tasks.ceph:[client] log to stderr = False 2021-03-09T02:47:52.136 INFO:tasks.ceph:[client] rbd default features = 125 2021-03-09T02:47:52.136 INFO:tasks.ceph:[client] rbd mirror image policy type = simple 2021-03-09T02:47:52.136 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.136 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.137 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.137 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.137 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.138 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.138 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.138 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.139 INFO:tasks.ceph:[global] mon client directed command retry = 5 2021-03-09T02:47:52.139 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2021-03-09T02:47:52.139 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-03-09T02:47:52.139 INFO:tasks.ceph:[mgr] debug ms = 1 2021-03-09T02:47:52.140 INFO:tasks.ceph:[mon] debug mon = 20 2021-03-09T02:47:52.140 INFO:tasks.ceph:[mon] debug ms = 1 2021-03-09T02:47:52.140 INFO:tasks.ceph:[mon] debug paxos = 20 2021-03-09T02:47:52.141 INFO:tasks.ceph:[osd] bdev async discard = True 2021-03-09T02:47:52.141 INFO:tasks.ceph:[osd] bdev enable discard = True 2021-03-09T02:47:52.141 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2021-03-09T02:47:52.141 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-03-09T02:47:52.142 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-03-09T02:47:52.142 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-03-09T02:47:52.142 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-03-09T02:47:52.143 INFO:tasks.ceph:[osd] debug filestore = 20 2021-03-09T02:47:52.143 INFO:tasks.ceph:[osd] debug journal = 20 2021-03-09T02:47:52.143 INFO:tasks.ceph:[osd] debug ms = 1 2021-03-09T02:47:52.143 INFO:tasks.ceph:[osd] debug osd = 25 2021-03-09T02:47:52.144 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-03-09T02:47:52.144 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-03-09T02:47:52.144 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-03-09T02:47:52.145 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-03-09T02:47:52.145 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-03-09T02:47:52.145 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-03-09T02:47:52.146 INFO:tasks.ceph:Setting up cluster1.mon.a... 2021-03-09T02:47:52.146 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster1.keyring 2021-03-09T02:47:52.207 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/cluster1.keyring 2021-03-09T02:47:52.209 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster1.keyring 2021-03-09T02:47:52.290 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-03-09T02:47:52.346 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster1.mon.a', '172.21.15.92')] 2021-03-09T02:47:52.347 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.92', 'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2021-03-09T02:47:52.348 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:47:52.348 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-03-09T02:47:52.393 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.92 --print /home/ubuntu/cephtest/cluster1.monmap 2021-03-09T02:47:52.466 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster1.monmap 2021-03-09T02:47:52.466 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: generated fsid 6274099e-850c-417b-b901-8a98ebef207c 2021-03-09T02:47:52.466 INFO:teuthology.orchestra.run.smithi092.stdout:epoch 0 2021-03-09T02:47:52.467 INFO:teuthology.orchestra.run.smithi092.stdout:fsid 6274099e-850c-417b-b901-8a98ebef207c 2021-03-09T02:47:52.467 INFO:teuthology.orchestra.run.smithi092.stdout:last_changed 2021-03-09T02:47:52.469634+0000 2021-03-09T02:47:52.467 INFO:teuthology.orchestra.run.smithi092.stdout:created 2021-03-09T02:47:52.469634+0000 2021-03-09T02:47:52.467 INFO:teuthology.orchestra.run.smithi092.stdout:min_mon_release 0 (unknown) 2021-03-09T02:47:52.468 INFO:teuthology.orchestra.run.smithi092.stdout:election_strategy: 1 2021-03-09T02:47:52.468 INFO:teuthology.orchestra.run.smithi092.stdout:0: [v2:172.21.15.92:3300/0,v1:172.21.15.92:6789/0] mon.a 2021-03-09T02:47:52.468 INFO:teuthology.orchestra.run.smithi092.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster1.monmap (1 monitors) 2021-03-09T02:47:52.470 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-03-09T02:47:52.517 INFO:tasks.ceph:Writing /etc/ceph/cluster1.conf for FSID 6274099e-850c-417b-b901-8a98ebef207c... 2021-03-09T02:47:52.519 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2021-03-09T02:47:52.561 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster1.conf && sudo chmod 0644 /etc/ceph/cluster1.conf > /dev/null 2021-03-09T02:47:52.590 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2021-03-09T02:47:52.590 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2021-03-09T02:47:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-03-09T02:47:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2021-03-09T02:47:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.591 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2021-03-09T02:47:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2021-03-09T02:47:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.592 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2021-03-09T02:47:52.593 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2021-03-09T02:47:52.593 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.593 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2021-03-09T02:47:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2021-03-09T02:47:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2021-03-09T02:47:52.594 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2021-03-09T02:47:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2021-03-09T02:47:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2021-03-09T02:47:52.595 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2021-03-09T02:47:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2021-03-09T02:47:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2021-03-09T02:47:52.596 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2021-03-09T02:47:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2021-03-09T02:47:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2021-03-09T02:47:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2021-03-09T02:47:52.597 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2021-03-09T02:47:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-03-09T02:47:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2021-03-09T02:47:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2021-03-09T02:47:52.598 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-03-09T02:47:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2021-03-09T02:47:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.599 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2021-03-09T02:47:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2021-03-09T02:47:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2021-03-09T02:47:52.600 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2021-03-09T02:47:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = 172.21.15.92 2021-03-09T02:47:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: mon client directed command retry = 5 2021-03-09T02:47:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject socket failures = 5000 2021-03-09T02:47:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = 6274099e-850c-417b-b901-8a98ebef207c 2021-03-09T02:47:52.601 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.602 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2021-03-09T02:47:52.602 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2021-03-09T02:47:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2021-03-09T02:47:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2021-03-09T02:47:52.603 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.604 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2021-03-09T02:47:52.604 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2021-03-09T02:47:52.604 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.604 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2021-03-09T02:47:52.604 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2021-03-09T02:47:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2021-03-09T02:47:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2021-03-09T02:47:52.605 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2021-03-09T02:47:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2021-03-09T02:47:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2021-03-09T02:47:52.606 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2021-03-09T02:47:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2021-03-09T02:47:52.607 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2021-03-09T02:47:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2021-03-09T02:47:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2021-03-09T02:47:52.608 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2021-03-09T02:47:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2021-03-09T02:47:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2021-03-09T02:47:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2021-03-09T02:47:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2021-03-09T02:47:52.609 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 20 2021-03-09T02:47:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 20 2021-03-09T02:47:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: debug filestore = 20 2021-03-09T02:47:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: debug journal = 20 2021-03-09T02:47:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2021-03-09T02:47:52.610 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 25 2021-03-09T02:47:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 10 2021-03-09T02:47:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2021-03-09T02:47:52.611 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2021-03-09T02:47:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2021-03-09T02:47:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2021-03-09T02:47:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2021-03-09T02:47:52.612 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.613 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2021-03-09T02:47:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2021-03-09T02:47:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2021-03-09T02:47:52.613 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2021-03-09T02:47:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2021-03-09T02:47:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2021-03-09T02:47:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2021-03-09T02:47:52.614 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2021-03-09T02:47:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.615 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2021-03-09T02:47:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2021-03-09T02:47:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2021-03-09T02:47:52.615 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2021-03-09T02:47:52.616 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2021-03-09T02:47:52.616 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2021-03-09T02:47:52.616 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2021-03-09T02:47:52.616 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2021-03-09T02:47:52.617 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2021-03-09T02:47:52.617 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2021-03-09T02:47:52.617 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2021-03-09T02:47:52.617 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:47:52.617 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2021-03-09T02:47:52.618 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2021-03-09T02:47:52.618 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2021-03-09T02:47:52.618 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2021-03-09T02:47:52.618 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-03-09T02:47:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-03-09T02:47:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: debug rbd = 20 2021-03-09T02:47:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: debug rbd_mirror = 15 2021-03-09T02:47:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: log to stderr = False 2021-03-09T02:47:52.619 INFO:teuthology.orchestra.run.smithi092.stdout: rbd default features = 125 2021-03-09T02:47:52.620 INFO:teuthology.orchestra.run.smithi092.stdout: rbd mirror image policy type = simple 2021-03-09T02:47:52.620 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2021-03-09T02:47:52.620 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.0] 2021-03-09T02:47:52.621 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.621 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.621 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.1] 2021-03-09T02:47:52.622 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.622 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.622 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.2] 2021-03-09T02:47:52.622 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.623 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.623 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.3] 2021-03-09T02:47:52.623 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.623 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.625 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2021-03-09T02:47:52.625 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2021-03-09T02:47:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-03-09T02:47:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2021-03-09T02:47:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.626 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2021-03-09T02:47:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2021-03-09T02:47:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2021-03-09T02:47:52.627 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2021-03-09T02:47:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2021-03-09T02:47:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2021-03-09T02:47:52.628 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2021-03-09T02:47:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2021-03-09T02:47:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.629 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2021-03-09T02:47:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2021-03-09T02:47:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.630 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2021-03-09T02:47:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2021-03-09T02:47:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2021-03-09T02:47:52.631 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2021-03-09T02:47:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2021-03-09T02:47:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2021-03-09T02:47:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2021-03-09T02:47:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2021-03-09T02:47:52.632 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-03-09T02:47:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2021-03-09T02:47:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2021-03-09T02:47:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.633 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-03-09T02:47:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2021-03-09T02:47:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.634 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2021-03-09T02:47:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2021-03-09T02:47:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2021-03-09T02:47:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2021-03-09T02:47:52.635 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = 172.21.15.92 2021-03-09T02:47:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: mon client directed command retry = 5 2021-03-09T02:47:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject socket failures = 5000 2021-03-09T02:47:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 6274099e-850c-417b-b901-8a98ebef207c 2021-03-09T02:47:52.636 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.637 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2021-03-09T02:47:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2021-03-09T02:47:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.637 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2021-03-09T02:47:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2021-03-09T02:47:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2021-03-09T02:47:52.638 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2021-03-09T02:47:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2021-03-09T02:47:52.639 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2021-03-09T02:47:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2021-03-09T02:47:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2021-03-09T02:47:52.640 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2021-03-09T02:47:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2021-03-09T02:47:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.641 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2021-03-09T02:47:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2021-03-09T02:47:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.642 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2021-03-09T02:47:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2021-03-09T02:47:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2021-03-09T02:47:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2021-03-09T02:47:52.643 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2021-03-09T02:47:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2021-03-09T02:47:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2021-03-09T02:47:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2021-03-09T02:47:52.644 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2021-03-09T02:47:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 20 2021-03-09T02:47:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 20 2021-03-09T02:47:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: debug filestore = 20 2021-03-09T02:47:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: debug journal = 20 2021-03-09T02:47:52.645 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2021-03-09T02:47:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 25 2021-03-09T02:47:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 10 2021-03-09T02:47:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2021-03-09T02:47:52.646 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2021-03-09T02:47:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2021-03-09T02:47:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2021-03-09T02:47:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2021-03-09T02:47:52.647 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.648 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2021-03-09T02:47:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2021-03-09T02:47:52.648 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2021-03-09T02:47:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2021-03-09T02:47:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2021-03-09T02:47:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2021-03-09T02:47:52.649 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2021-03-09T02:47:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2021-03-09T02:47:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.650 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2021-03-09T02:47:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2021-03-09T02:47:52.650 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2021-03-09T02:47:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2021-03-09T02:47:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2021-03-09T02:47:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2021-03-09T02:47:52.651 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2021-03-09T02:47:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2021-03-09T02:47:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2021-03-09T02:47:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2021-03-09T02:47:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2021-03-09T02:47:52.652 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:47:52.653 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2021-03-09T02:47:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2021-03-09T02:47:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2021-03-09T02:47:52.653 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2021-03-09T02:47:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-03-09T02:47:52.654 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-03-09T02:47:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: debug rbd = 20 2021-03-09T02:47:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: debug rbd_mirror = 15 2021-03-09T02:47:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: log to stderr = False 2021-03-09T02:47:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: rbd default features = 125 2021-03-09T02:47:52.655 INFO:teuthology.orchestra.run.smithi144.stdout: rbd mirror image policy type = simple 2021-03-09T02:47:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2021-03-09T02:47:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.0] 2021-03-09T02:47:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.657 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.1] 2021-03-09T02:47:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.657 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.2] 2021-03-09T02:47:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.658 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.3] 2021-03-09T02:47:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:47:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:47:52.659 INFO:tasks.ceph:Creating admin key on cluster1.mon.a... 2021-03-09T02:47:52.660 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster1.keyring 2021-03-09T02:47:52.713 INFO:tasks.ceph:Copying monmap to all nodes... 2021-03-09T02:47:52.713 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:47:52.713 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/cluster1.keyring of=/dev/stdout 2021-03-09T02:47:52.720 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:47:52.720 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/home/ubuntu/cephtest/cluster1.monmap of=/dev/stdout 2021-03-09T02:47:52.769 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2021-03-09T02:47:52.770 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:47:52.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/cluster1.keyring 2021-03-09T02:47:52.771 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-03-09T02:47:52.837 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:47:52.837 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2021-03-09T02:47:52.886 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2021-03-09T02:47:52.887 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:47:52.887 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/cluster1.keyring 2021-03-09T02:47:52.887 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/cluster1.keyring 2021-03-09T02:47:52.912 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:47:52.912 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/cluster1.monmap 2021-03-09T02:47:52.966 INFO:tasks.ceph:Setting up mon nodes... 2021-03-09T02:47:52.966 INFO:tasks.ceph:Setting up mgr nodes... 2021-03-09T02:47:52.967 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/cluster1-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster1-x/keyring 2021-03-09T02:47:53.011 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/cluster1-x/keyring 2021-03-09T02:47:53.013 INFO:tasks.ceph:Setting up mds nodes... 2021-03-09T02:47:53.014 INFO:tasks.ceph_client:Setting up client nodes... 2021-03-09T02:47:53.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster1.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.0.keyring 2021-03-09T02:47:53.088 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/cluster1.client.0.keyring 2021-03-09T02:47:53.101 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster1.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.keyring 2021-03-09T02:47:53.135 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster1.client.mirror.keyring 2021-03-09T02:47:53.148 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster1.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.0.keyring 2021-03-09T02:47:53.228 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster1.client.mirror.0.keyring 2021-03-09T02:47:53.240 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster1.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.1.keyring 2021-03-09T02:47:53.317 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster1.client.mirror.1.keyring 2021-03-09T02:47:53.328 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster1.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.2.keyring 2021-03-09T02:47:53.403 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster1.client.mirror.2.keyring 2021-03-09T02:47:53.416 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster1.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.3.keyring 2021-03-09T02:47:53.495 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster1.client.mirror.3.keyring 2021-03-09T02:47:53.507 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster1.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.4.keyring 2021-03-09T02:47:53.580 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster1.client.mirror.4.keyring 2021-03-09T02:47:53.593 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster1.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.5.keyring 2021-03-09T02:47:53.668 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster1.client.mirror.5.keyring 2021-03-09T02:47:53.680 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster1.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster1.client.mirror.6.keyring 2021-03-09T02:47:53.756 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster1.client.mirror.6.keyring 2021-03-09T02:47:53.768 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-03-09T02:47:53.769 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}} 2021-03-09T02:47:53.770 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/cluster1-0 2021-03-09T02:47:53.785 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-09T02:47:53.786 INFO:tasks.ceph:role: cluster1.osd.0 2021-03-09T02:47:53.786 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com 2021-03-09T02:47:53.786 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-03-09T02:47:54.315 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-03-09T02:47:54.316 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-09T02:47:54.316 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-09T02:47:54.316 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-09T02:47:54.317 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2021-03-09T02:47:54.317 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-09T02:47:54.317 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-09T02:47:54.317 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-09T02:47:54.318 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-09T02:47:54.319 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2021-03-09T02:47:54.319 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster1-0 2021-03-09T02:47:54.411 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-0 2021-03-09T02:47:54.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:47:54.423 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2021-03-09T02:47:54.423 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/cluster1-1 2021-03-09T02:47:54.480 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-09T02:47:54.480 INFO:tasks.ceph:role: cluster1.osd.1 2021-03-09T02:47:54.480 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com 2021-03-09T02:47:54.481 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-03-09T02:47:55.008 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-03-09T02:47:55.009 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-09T02:47:55.009 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-09T02:47:55.010 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-09T02:47:55.010 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2021-03-09T02:47:55.010 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-09T02:47:55.011 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-09T02:47:55.011 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-09T02:47:55.012 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-09T02:47:55.013 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2021-03-09T02:47:55.014 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster1-1 2021-03-09T02:47:55.054 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-1 2021-03-09T02:47:55.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:47:55.107 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2021-03-09T02:47:55.108 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/osd/cluster1-2 2021-03-09T02:47:55.170 INFO:tasks.ceph:roles_to_devs: {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-09T02:47:55.171 INFO:tasks.ceph:role: cluster1.osd.2 2021-03-09T02:47:55.171 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com 2021-03-09T02:47:55.171 DEBUG:teuthology.orchestra.run.smithi092:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-03-09T02:47:55.692 INFO:teuthology.orchestra.run.smithi092.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-03-09T02:47:55.692 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-09T02:47:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-09T02:47:55.693 INFO:teuthology.orchestra.run.smithi092.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-09T02:47:55.693 INFO:teuthology.orchestra.run.smithi092.stdout: = sunit=0 swidth=0 blks 2021-03-09T02:47:55.693 INFO:teuthology.orchestra.run.smithi092.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-09T02:47:55.694 INFO:teuthology.orchestra.run.smithi092.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-09T02:47:55.694 INFO:teuthology.orchestra.run.smithi092.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-09T02:47:55.694 INFO:teuthology.orchestra.run.smithi092.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-09T02:47:55.696 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi092.front.sepia.ceph.com -o noatime 2021-03-09T02:47:55.696 DEBUG:teuthology.orchestra.run.smithi092:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster1-2 2021-03-09T02:47:55.739 DEBUG:teuthology.orchestra.run.smithi092:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster1-2 2021-03-09T02:47:55.795 INFO:teuthology.orchestra.run.smithi092.stderr:sudo: /sbin/restorecon: command not found 2021-03-09T02:47:55.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:47:55.796 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-03-09T02:47:55.910 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:55.910+0000 7f7cb6c4d080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-0/keyring: can't open /var/lib/ceph/osd/cluster1-0/keyring: (2) No such file or directory 2021-03-09T02:47:55.912 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:55.914+0000 7f7cb6c4d080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-0/keyring 2021-03-09T02:47:55.912 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:55.914+0000 7f7cb6c4d080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-03-09T02:47:55.913 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:55.914+0000 7f7cb6c4d080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-03-09T02:47:55.913 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:55.914+0000 7f7cb6c4d080 -1 bluestore(/var/lib/ceph/osd/cluster1-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-0/block: (2) No such file or directory 2021-03-09T02:47:55.917 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:55.918+0000 7f7cb6c4d080 -1 bluestore(/var/lib/ceph/osd/cluster1-0) _read_fsid unparsable uuid 2021-03-09T02:47:58.865 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-0 2021-03-09T02:47:58.883 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-03-09T02:47:58.962 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:58.962+0000 7f76bbe16080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-1/keyring: can't open /var/lib/ceph/osd/cluster1-1/keyring: (2) No such file or directory 2021-03-09T02:47:58.963 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:58.962+0000 7f76bbe16080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-1/keyring 2021-03-09T02:47:58.963 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:58.962+0000 7f76bbe16080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-03-09T02:47:58.963 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:58.962+0000 7f76bbe16080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-03-09T02:47:58.964 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:58.962+0000 7f76bbe16080 -1 bluestore(/var/lib/ceph/osd/cluster1-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-1/block: (2) No such file or directory 2021-03-09T02:47:58.965 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:47:58.962+0000 7f76bbe16080 -1 bluestore(/var/lib/ceph/osd/cluster1-1) _read_fsid unparsable uuid 2021-03-09T02:48:01.911 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-1 2021-03-09T02:48:01.971 DEBUG:teuthology.orchestra.run.smithi092:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster1 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster1.monmap 2021-03-09T02:48:02.049 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:48:02.050+0000 7fd00ee6e080 -1 auth: error reading file: /var/lib/ceph/osd/cluster1-2/keyring: can't open /var/lib/ceph/osd/cluster1-2/keyring: (2) No such file or directory 2021-03-09T02:48:02.049 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:48:02.050+0000 7fd00ee6e080 -1 created new key in keyring /var/lib/ceph/osd/cluster1-2/keyring 2021-03-09T02:48:02.049 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:48:02.050+0000 7fd00ee6e080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-03-09T02:48:02.050 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:48:02.050+0000 7fd00ee6e080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-03-09T02:48:02.050 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:48:02.050+0000 7fd00ee6e080 -1 bluestore(/var/lib/ceph/osd/cluster1-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster1-2/block: (2) No such file or directory 2021-03-09T02:48:02.050 INFO:teuthology.orchestra.run.smithi092.stderr:2021-03-09T02:48:02.050+0000 7fd00ee6e080 -1 bluestore(/var/lib/ceph/osd/cluster1-2) _read_fsid unparsable uuid 2021-03-09T02:48:05.011 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster1-2 2021-03-09T02:48:05.071 INFO:tasks.ceph:Reading keys from all nodes... 2021-03-09T02:48:05.072 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:05.072 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/cluster1-x/keyring of=/dev/stdout 2021-03-09T02:48:05.129 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:05.130 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/cluster1-0/keyring of=/dev/stdout 2021-03-09T02:48:05.185 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:05.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/cluster1-1/keyring of=/dev/stdout 2021-03-09T02:48:05.246 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:05.246 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/cluster1-2/keyring of=/dev/stdout 2021-03-09T02:48:05.302 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:05.302 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/cluster1.client.0.keyring of=/dev/stdout 2021-03-09T02:48:05.349 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:05.350 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster1.client.mirror.keyring of=/dev/stdout 2021-03-09T02:48:05.357 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:05.357 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster1.client.mirror.0.keyring of=/dev/stdout 2021-03-09T02:48:05.405 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:05.406 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster1.client.mirror.1.keyring of=/dev/stdout 2021-03-09T02:48:05.453 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:05.454 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster1.client.mirror.2.keyring of=/dev/stdout 2021-03-09T02:48:05.501 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:05.502 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster1.client.mirror.3.keyring of=/dev/stdout 2021-03-09T02:48:05.549 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:05.550 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster1.client.mirror.4.keyring of=/dev/stdout 2021-03-09T02:48:05.597 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:05.598 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster1.client.mirror.5.keyring of=/dev/stdout 2021-03-09T02:48:05.645 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:05.646 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster1.client.mirror.6.keyring of=/dev/stdout 2021-03-09T02:48:05.693 INFO:tasks.ceph:Adding keys to all mons... 2021-03-09T02:48:05.694 DEBUG:teuthology.orchestra.run.smithi092:> sudo tee -a /etc/ceph/cluster1.keyring 2021-03-09T02:48:05.708 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr.x] 2021-03-09T02:48:05.708 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDZ4UZgF3/nABAADhucT7hiww69lIesk6/ycg== 2021-03-09T02:48:05.708 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.0] 2021-03-09T02:48:05.708 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDb4UZgTQt4NhAAD9+xx5PkrhUUpYvEsj2xFA== 2021-03-09T02:48:05.709 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.1] 2021-03-09T02:48:05.709 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDe4UZgY2CEORAA02rrAZ7uS4YjmoamDcD4vg== 2021-03-09T02:48:05.709 INFO:teuthology.orchestra.run.smithi092.stdout:[osd.2] 2021-03-09T02:48:05.709 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDi4UZgRechAxAAX8HMLZJB5O2JzuElcvOLSw== 2021-03-09T02:48:05.710 INFO:teuthology.orchestra.run.smithi092.stdout:[client.0] 2021-03-09T02:48:05.710 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDZ4UZgLMuBBRAAfC/6+7PaJlMspILjnC3inA== 2021-03-09T02:48:05.710 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror] 2021-03-09T02:48:05.710 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDZ4UZg1w0iCBAAKP8ZNYZoSUi194T/IWGA7Q== 2021-03-09T02:48:05.710 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.0] 2021-03-09T02:48:05.711 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDZ4UZgqCGqDRAAGKvVRTAGYQxMTkFrnDOMaw== 2021-03-09T02:48:05.711 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.1] 2021-03-09T02:48:05.711 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDZ4UZg8PfxEhAArfJxR4ikZVowl4z/aG904A== 2021-03-09T02:48:05.712 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.2] 2021-03-09T02:48:05.712 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDZ4UZgmEkbGBAAwOVl+Ovle8fZBvNqujTuQw== 2021-03-09T02:48:05.712 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.3] 2021-03-09T02:48:05.712 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDZ4UZgdhCZHRAASOpcgkDop0bIfLbRktBUsg== 2021-03-09T02:48:05.713 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.4] 2021-03-09T02:48:05.713 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDZ4UZg5HGpIhAAj5WthEzH0xSYJZ2z8lMxnQ== 2021-03-09T02:48:05.713 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.5] 2021-03-09T02:48:05.714 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDZ4UZgcQnpJxAATRtHjAH/9cLiqRw/huOcEw== 2021-03-09T02:48:05.714 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.6] 2021-03-09T02:48:05.714 INFO:teuthology.orchestra.run.smithi092.stdout: key = AQDZ4UZgRL8lLRAA3VZAZx2xx+gAgyZZ/2WD4A== 2021-03-09T02:48:05.715 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-03-09T02:48:05.801 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-09T02:48:05.886 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-09T02:48:05.969 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-09T02:48:06.055 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:06.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:06.185 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:06.272 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:06.312 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:06.401 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:06.447 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:06.532 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:06.575 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster1.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:06.664 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-03-09T02:48:06.664 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mon/cluster1-a 2021-03-09T02:48:06.722 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster1 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster1.monmap --keyring /etc/ceph/cluster1.keyring 2021-03-09T02:48:06.818 DEBUG:teuthology.orchestra.run.smithi092:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster1-a 2021-03-09T02:48:06.833 DEBUG:teuthology.orchestra.run.smithi092:> rm -- /home/ubuntu/cephtest/cluster1.monmap 2021-03-09T02:48:06.881 INFO:tasks.ceph:Starting mon daemons in cluster cluster1... 2021-03-09T02:48:06.881 INFO:tasks.ceph.cluster1.mon.a:Restarting daemon 2021-03-09T02:48:06.882 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster1 -i a 2021-03-09T02:48:06.925 INFO:tasks.ceph.cluster1.mon.a:Started 2021-03-09T02:48:06.926 INFO:tasks.ceph:Starting mgr daemons in cluster cluster1... 2021-03-09T02:48:06.926 INFO:tasks.ceph.cluster1.mgr.x:Restarting daemon 2021-03-09T02:48:06.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster1 -i x 2021-03-09T02:48:06.929 INFO:tasks.ceph.cluster1.mgr.x:Started 2021-03-09T02:48:06.929 INFO:tasks.ceph:Setting crush tunables to default 2021-03-09T02:48:06.930 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster cluster1 osd crush tunables default 2021-03-09T02:48:07.272 INFO:teuthology.orchestra.run.smithi092.stderr:adjusted tunables profile to default 2021-03-09T02:48:07.284 INFO:tasks.ceph:Starting osd daemons in cluster cluster1... 2021-03-09T02:48:07.285 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:07.286 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/cluster1-0/fsid of=/dev/stdout 2021-03-09T02:48:07.296 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:07.296 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/cluster1-1/fsid of=/dev/stdout 2021-03-09T02:48:07.353 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:07.354 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/osd/cluster1-2/fsid of=/dev/stdout 2021-03-09T02:48:07.409 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster cluster1 osd new 64fd19a3-799d-442a-b77f-6ee7c69d4009 0 2021-03-09T02:48:07.670 INFO:teuthology.orchestra.run.smithi092.stdout:0 2021-03-09T02:48:07.684 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster cluster1 osd new 6f7bcdd8-b172-4534-8643-90f8d130e9ed 1 2021-03-09T02:48:07.904 INFO:teuthology.orchestra.run.smithi092.stdout:1 2021-03-09T02:48:07.917 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster cluster1 osd new b07a806e-aabd-4c83-8407-a439f93af83e 2 2021-03-09T02:48:08.130 INFO:teuthology.orchestra.run.smithi092.stdout:2 2021-03-09T02:48:08.143 INFO:tasks.ceph.cluster1.osd.0:Restarting daemon 2021-03-09T02:48:08.144 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 0 2021-03-09T02:48:08.147 INFO:tasks.ceph.cluster1.osd.0:Started 2021-03-09T02:48:08.147 INFO:tasks.ceph.cluster1.osd.1:Restarting daemon 2021-03-09T02:48:08.147 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 1 2021-03-09T02:48:08.149 INFO:tasks.ceph.cluster1.osd.1:Started 2021-03-09T02:48:08.150 INFO:tasks.ceph.cluster1.osd.2:Restarting daemon 2021-03-09T02:48:08.151 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster1 -i 2 2021-03-09T02:48:08.153 INFO:tasks.ceph.cluster1.osd.2:Started 2021-03-09T02:48:08.154 INFO:tasks.ceph:Waiting for OSDs to come up 2021-03-09T02:48:08.456 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2021-03-09T02:48:08.505 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T02:48:08.506+0000 7f9031128080 -1 Falling back to public interface 2021-03-09T02:48:08.650 INFO:teuthology.misc.health.smithi092.stdout: 2021-03-09T02:48:08.650 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":5,"fsid":"6274099e-850c-417b-b901-8a98ebef207c","created":"2021-03-09T02:48:07.043135+0000","modified":"2021-03-09T02:48:08.131582+0000","last_up_change":"0.000000","last_in_change":"2021-03-09T02:48:08.131582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"64fd19a3-799d-442a-b77f-6ee7c69d4009","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6f7bcdd8-b172-4534-8643-90f8d130e9ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b07a806e-aabd-4c83-8407-a439f93af83e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-03-09T02:48:08.662 DEBUG:teuthology.misc:0 of 3 OSDs are up 2021-03-09T02:48:08.764 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T02:48:08.766+0000 7fddbb699080 -1 Falling back to public interface 2021-03-09T02:48:08.765 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T02:48:08.766+0000 7fa0a1948080 -1 Falling back to public interface 2021-03-09T02:48:08.766 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T02:48:08.766+0000 7fddb6369700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6810/0: (98) Address already in use 2021-03-09T02:48:12.316 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T02:48:12.318+0000 7f9031128080 -1 osd.2 0 log_to_monitors {default=true} 2021-03-09T02:48:12.505 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T02:48:12.506+0000 7fddbb699080 -1 osd.1 0 log_to_monitors {default=true} 2021-03-09T02:48:12.572 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T02:48:12.574+0000 7fa0a1948080 -1 osd.0 0 log_to_monitors {default=true} 2021-03-09T02:48:14.138 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T02:48:14.138+0000 7fddb2b62700 -1 osd.1 0 waiting for initial osdmap 2021-03-09T02:48:14.138 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T02:48:14.138+0000 7f90285f1700 -1 osd.2 0 waiting for initial osdmap 2021-03-09T02:48:14.139 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T02:48:14.138+0000 7fa098e11700 -1 osd.0 0 waiting for initial osdmap 2021-03-09T02:48:14.965 DEBUG:teuthology.orchestra.run.smithi092:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster1 osd dump --format=json 2021-03-09T02:48:15.270 INFO:teuthology.misc.health.smithi092.stdout: 2021-03-09T02:48:15.270 INFO:teuthology.misc.health.smithi092.stdout:{"epoch":8,"fsid":"6274099e-850c-417b-b901-8a98ebef207c","created":"2021-03-09T02:48:07.043135+0000","modified":"2021-03-09T02:48:15.135200+0000","last_up_change":"2021-03-09T02:48:15.135200+0000","last_in_change":"2021-03-09T02:48:08.131582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"64fd19a3-799d-442a-b77f-6ee7c69d4009","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":16786},{"type":"v1","addr":"172.21.15.92:6812","nonce":16786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":16786},{"type":"v1","addr":"172.21.15.92:6816","nonce":16786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":16786},{"type":"v1","addr":"172.21.15.92:6823","nonce":16786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":16786},{"type":"v1","addr":"172.21.15.92:6820","nonce":16786}]},"public_addr":"172.21.15.92:6812/16786","cluster_addr":"172.21.15.92:6816/16786","heartbeat_back_addr":"172.21.15.92:6823/16786","heartbeat_front_addr":"172.21.15.92:6820/16786","state":["exists","up"]},{"osd":1,"uuid":"6f7bcdd8-b172-4534-8643-90f8d130e9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":16784},{"type":"v1","addr":"172.21.15.92:6809","nonce":16784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":16784},{"type":"v1","addr":"172.21.15.92:6813","nonce":16784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":16784},{"type":"v1","addr":"172.21.15.92:6821","nonce":16784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":16784},{"type":"v1","addr":"172.21.15.92:6817","nonce":16784}]},"public_addr":"172.21.15.92:6809/16784","cluster_addr":"172.21.15.92:6813/16784","heartbeat_back_addr":"172.21.15.92:6821/16784","heartbeat_front_addr":"172.21.15.92:6817/16784","state":["exists","up"]},{"osd":2,"uuid":"b07a806e-aabd-4c83-8407-a439f93af83e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":16785},{"type":"v1","addr":"172.21.15.92:6801","nonce":16785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":16785},{"type":"v1","addr":"172.21.15.92:6803","nonce":16785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":16785},{"type":"v1","addr":"172.21.15.92:6807","nonce":16785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":16785},{"type":"v1","addr":"172.21.15.92:6805","nonce":16785}]},"public_addr":"172.21.15.92:6801/16785","cluster_addr":"172.21.15.92:6803/16785","heartbeat_back_addr":"172.21.15.92:6807/16785","heartbeat_front_addr":"172.21.15.92:6805/16785","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-03-09T02:48:15.288 DEBUG:teuthology.misc:3 of 3 OSDs are up 2021-03-09T02:48:15.289 INFO:tasks.ceph:Creating RBD pool 2021-03-09T02:48:15.289 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster cluster1 osd pool create rbd 8 2021-03-09T02:48:16.152 INFO:teuthology.orchestra.run.smithi092.stderr:pool 'rbd' created 2021-03-09T02:48:16.165 DEBUG:teuthology.orchestra.run.smithi092:> sudo ceph --cluster cluster1 osd pool application enable rbd rbd --yes-i-really-mean-it 2021-03-09T02:48:17.174 INFO:teuthology.orchestra.run.smithi092.stderr:enabled application 'rbd' on pool 'rbd' 2021-03-09T02:48:17.187 INFO:tasks.ceph:Starting mds daemons in cluster cluster1... 2021-03-09T02:48:17.188 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-03-09T02:48:17.189 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-03-09T02:48:17.494 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:17.494 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":10,"fsid":"6274099e-850c-417b-b901-8a98ebef207c","created":"2021-03-09T02:48:07.043135+0000","modified":"2021-03-09T02:48:17.151484+0000","last_up_change":"2021-03-09T02:48:15.135200+0000","last_in_change":"2021-03-09T02:48:08.131582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-09T02:48:15.201727+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-09T02:48:15.656374+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"64fd19a3-799d-442a-b77f-6ee7c69d4009","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":16786},{"type":"v1","addr":"172.21.15.92:6812","nonce":16786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":16786},{"type":"v1","addr":"172.21.15.92:6816","nonce":16786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":16786},{"type":"v1","addr":"172.21.15.92:6823","nonce":16786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":16786},{"type":"v1","addr":"172.21.15.92:6820","nonce":16786}]},"public_addr":"172.21.15.92:6812/16786","cluster_addr":"172.21.15.92:6816/16786","heartbeat_back_addr":"172.21.15.92:6823/16786","heartbeat_front_addr":"172.21.15.92:6820/16786","state":["exists","up"]},{"osd":1,"uuid":"6f7bcdd8-b172-4534-8643-90f8d130e9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":16784},{"type":"v1","addr":"172.21.15.92:6809","nonce":16784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":16784},{"type":"v1","addr":"172.21.15.92:6813","nonce":16784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":16784},{"type":"v1","addr":"172.21.15.92:6821","nonce":16784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":16784},{"type":"v1","addr":"172.21.15.92:6817","nonce":16784}]},"public_addr":"172.21.15.92:6809/16784","cluster_addr":"172.21.15.92:6813/16784","heartbeat_back_addr":"172.21.15.92:6821/16784","heartbeat_front_addr":"172.21.15.92:6817/16784","state":["exists","up"]},{"osd":2,"uuid":"b07a806e-aabd-4c83-8407-a439f93af83e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":16785},{"type":"v1","addr":"172.21.15.92:6801","nonce":16785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":16785},{"type":"v1","addr":"172.21.15.92:6803","nonce":16785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":16785},{"type":"v1","addr":"172.21.15.92:6807","nonce":16785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":16785},{"type":"v1","addr":"172.21.15.92:6805","nonce":16785}]},"public_addr":"172.21.15.92:6801/16785","cluster_addr":"172.21.15.92:6803/16785","heartbeat_back_addr":"172.21.15.92:6807/16785","heartbeat_front_addr":"172.21.15.92:6805/16785","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:13.532731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:13.560470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:13.353749+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-03-09T02:48:17.516 INFO:tasks.ceph.ceph_manager.cluster1:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-03-09T02:48:15.201727+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-03-09T02:48:15.656374+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-03-09T02:48:17.517 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd pool get device_health_metrics pg_num 2021-03-09T02:48:17.818 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 1 2021-03-09T02:48:17.833 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd pool get rbd pg_num 2021-03-09T02:48:18.135 INFO:teuthology.orchestra.run.smithi092.stdout:pg_num: 8 2021-03-09T02:48:18.154 INFO:tasks.ceph:Waiting until cluster1 daemons up and pgs clean... 2021-03-09T02:48:18.155 INFO:tasks.ceph.ceph_manager.cluster1:waiting for mgr available 2021-03-09T02:48:18.156 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 mgr dump --format=json 2021-03-09T02:48:18.491 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:18.504 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":4,"active_gid":4099,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6824","nonce":16600},{"type":"v1","addr":"172.21.15.92:6825","nonce":16600}]},"active_addr":"172.21.15.92:6825/16600","active_change":"2021-03-09T02:48:11.832783+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3523312907}]},{"addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":2293306424}]}]}} 2021-03-09T02:48:18.507 INFO:tasks.ceph.ceph_manager.cluster1:mgr available! 2021-03-09T02:48:18.508 INFO:tasks.ceph.ceph_manager.cluster1:waiting for all up 2021-03-09T02:48:18.508 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-03-09T02:48:18.797 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:18.798 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":11,"fsid":"6274099e-850c-417b-b901-8a98ebef207c","created":"2021-03-09T02:48:07.043135+0000","modified":"2021-03-09T02:48:18.155903+0000","last_up_change":"2021-03-09T02:48:15.135200+0000","last_in_change":"2021-03-09T02:48:08.131582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-09T02:48:15.201727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-09T02:48:15.656374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"64fd19a3-799d-442a-b77f-6ee7c69d4009","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":16786},{"type":"v1","addr":"172.21.15.92:6812","nonce":16786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":16786},{"type":"v1","addr":"172.21.15.92:6816","nonce":16786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":16786},{"type":"v1","addr":"172.21.15.92:6823","nonce":16786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":16786},{"type":"v1","addr":"172.21.15.92:6820","nonce":16786}]},"public_addr":"172.21.15.92:6812/16786","cluster_addr":"172.21.15.92:6816/16786","heartbeat_back_addr":"172.21.15.92:6823/16786","heartbeat_front_addr":"172.21.15.92:6820/16786","state":["exists","up"]},{"osd":1,"uuid":"6f7bcdd8-b172-4534-8643-90f8d130e9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":16784},{"type":"v1","addr":"172.21.15.92:6809","nonce":16784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":16784},{"type":"v1","addr":"172.21.15.92:6813","nonce":16784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":16784},{"type":"v1","addr":"172.21.15.92:6821","nonce":16784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":16784},{"type":"v1","addr":"172.21.15.92:6817","nonce":16784}]},"public_addr":"172.21.15.92:6809/16784","cluster_addr":"172.21.15.92:6813/16784","heartbeat_back_addr":"172.21.15.92:6821/16784","heartbeat_front_addr":"172.21.15.92:6817/16784","state":["exists","up"]},{"osd":2,"uuid":"b07a806e-aabd-4c83-8407-a439f93af83e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":16785},{"type":"v1","addr":"172.21.15.92:6801","nonce":16785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":16785},{"type":"v1","addr":"172.21.15.92:6803","nonce":16785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":16785},{"type":"v1","addr":"172.21.15.92:6807","nonce":16785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":16785},{"type":"v1","addr":"172.21.15.92:6805","nonce":16785}]},"public_addr":"172.21.15.92:6801/16785","cluster_addr":"172.21.15.92:6803/16785","heartbeat_back_addr":"172.21.15.92:6807/16785","heartbeat_front_addr":"172.21.15.92:6805/16785","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:13.532731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:13.560470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:13.353749+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-03-09T02:48:18.812 INFO:tasks.ceph.ceph_manager.cluster1:all up! 2021-03-09T02:48:18.812 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd dump --format=json 2021-03-09T02:48:19.113 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:19.114 INFO:teuthology.orchestra.run.smithi092.stdout:{"epoch":11,"fsid":"6274099e-850c-417b-b901-8a98ebef207c","created":"2021-03-09T02:48:07.043135+0000","modified":"2021-03-09T02:48:18.155903+0000","last_up_change":"2021-03-09T02:48:15.135200+0000","last_in_change":"2021-03-09T02:48:08.131582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-09T02:48:15.201727+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-09T02:48:15.656374+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"64fd19a3-799d-442a-b77f-6ee7c69d4009","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6810","nonce":16786},{"type":"v1","addr":"172.21.15.92:6812","nonce":16786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6814","nonce":16786},{"type":"v1","addr":"172.21.15.92:6816","nonce":16786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6822","nonce":16786},{"type":"v1","addr":"172.21.15.92:6823","nonce":16786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6818","nonce":16786},{"type":"v1","addr":"172.21.15.92:6820","nonce":16786}]},"public_addr":"172.21.15.92:6812/16786","cluster_addr":"172.21.15.92:6816/16786","heartbeat_back_addr":"172.21.15.92:6823/16786","heartbeat_front_addr":"172.21.15.92:6820/16786","state":["exists","up"]},{"osd":1,"uuid":"6f7bcdd8-b172-4534-8643-90f8d130e9ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6808","nonce":16784},{"type":"v1","addr":"172.21.15.92:6809","nonce":16784}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6811","nonce":16784},{"type":"v1","addr":"172.21.15.92:6813","nonce":16784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6819","nonce":16784},{"type":"v1","addr":"172.21.15.92:6821","nonce":16784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6815","nonce":16784},{"type":"v1","addr":"172.21.15.92:6817","nonce":16784}]},"public_addr":"172.21.15.92:6809/16784","cluster_addr":"172.21.15.92:6813/16784","heartbeat_back_addr":"172.21.15.92:6821/16784","heartbeat_front_addr":"172.21.15.92:6817/16784","state":["exists","up"]},{"osd":2,"uuid":"b07a806e-aabd-4c83-8407-a439f93af83e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6800","nonce":16785},{"type":"v1","addr":"172.21.15.92:6801","nonce":16785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6802","nonce":16785},{"type":"v1","addr":"172.21.15.92:6803","nonce":16785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6806","nonce":16785},{"type":"v1","addr":"172.21.15.92:6807","nonce":16785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6804","nonce":16785},{"type":"v1","addr":"172.21.15.92:6805","nonce":16785}]},"public_addr":"172.21.15.92:6801/16785","cluster_addr":"172.21.15.92:6803/16785","heartbeat_back_addr":"172.21.15.92:6807/16785","heartbeat_front_addr":"172.21.15.92:6805/16785","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:13.532731+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:13.560470+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:13.353749+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-03-09T02:48:19.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.0 flush_pg_stats 2021-03-09T02:48:19.298 INFO:teuthology.orchestra.run.smithi092.stdout:34359738371 2021-03-09T02:48:19.299 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.1 flush_pg_stats 2021-03-09T02:48:19.477 INFO:teuthology.orchestra.run.smithi092.stdout:34359738371 2021-03-09T02:48:19.478 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 tell osd.2 flush_pg_stats 2021-03-09T02:48:19.658 INFO:teuthology.orchestra.run.smithi092.stdout:34359738370 2021-03-09T02:48:19.658 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2021-03-09T02:48:19.964 INFO:teuthology.orchestra.run.smithi092.stdout:34359738370 2021-03-09T02:48:19.978 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738370 for osd.0 2021-03-09T02:48:20.980 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.0 2021-03-09T02:48:21.283 INFO:teuthology.orchestra.run.smithi092.stdout:34359738371 2021-03-09T02:48:21.298 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738371 for osd.0 2021-03-09T02:48:21.298 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.1 2021-03-09T02:48:21.598 INFO:teuthology.orchestra.run.smithi092.stdout:34359738371 2021-03-09T02:48:21.616 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738371 got 34359738371 for osd.1 2021-03-09T02:48:21.617 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 osd last-stat-seq osd.2 2021-03-09T02:48:21.909 INFO:teuthology.orchestra.run.smithi092.stdout:34359738370 2021-03-09T02:48:21.923 INFO:tasks.ceph.ceph_manager.cluster1:need seq 34359738370 got 34359738370 for osd.2 2021-03-09T02:48:21.924 INFO:tasks.ceph.ceph_manager.cluster1:waiting for clean 2021-03-09T02:48:21.924 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2021-03-09T02:48:22.216 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:22.218 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2021-03-09T02:48:22.230 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-03-09T02:48:21.841412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14648,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100872,"statfs":{"total":289910292480,"available":289895292928,"internally_reserved":0,"allocated":294912,"data_stored":80124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.174649+0000","last_change":"2021-03-09T02:48:17.173198+0000","last_active":"2021-03-09T02:48:17.174649+0000","last_peered":"2021-03-09T02:48:17.174649+0000","last_clean":"2021-03-09T02:48:17.174649+0000","last_became_active":"2021-03-09T02:48:17.172725+0000","last_became_peered":"2021-03-09T02:48:17.172725+0000","last_unstale":"2021-03-09T02:48:17.174649+0000","last_undegraded":"2021-03-09T02:48:17.174649+0000","last_fullsized":"2021-03-09T02:48:17.174649+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.177042+0000","last_change":"2021-03-09T02:48:17.175331+0000","last_active":"2021-03-09T02:48:17.177042+0000","last_peered":"2021-03-09T02:48:17.177042+0000","last_clean":"2021-03-09T02:48:17.177042+0000","last_became_active":"2021-03-09T02:48:17.174657+0000","last_became_peered":"2021-03-09T02:48:17.174657+0000","last_unstale":"2021-03-09T02:48:17.177042+0000","last_undegraded":"2021-03-09T02:48:17.177042+0000","last_fullsized":"2021-03-09T02:48:17.177042+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.174578+0000","last_change":"2021-03-09T02:48:17.173195+0000","last_active":"2021-03-09T02:48:17.174578+0000","last_peered":"2021-03-09T02:48:17.174578+0000","last_clean":"2021-03-09T02:48:17.174578+0000","last_became_active":"2021-03-09T02:48:17.171539+0000","last_became_peered":"2021-03-09T02:48:17.171539+0000","last_unstale":"2021-03-09T02:48:17.174578+0000","last_undegraded":"2021-03-09T02:48:17.174578+0000","last_fullsized":"2021-03-09T02:48:17.174578+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.175888+0000","last_change":"2021-03-09T02:48:17.174593+0000","last_active":"2021-03-09T02:48:17.175888+0000","last_peered":"2021-03-09T02:48:17.175888+0000","last_clean":"2021-03-09T02:48:17.175888+0000","last_became_active":"2021-03-09T02:48:17.174205+0000","last_became_peered":"2021-03-09T02:48:17.174205+0000","last_unstale":"2021-03-09T02:48:17.175888+0000","last_undegraded":"2021-03-09T02:48:17.175888+0000","last_fullsized":"2021-03-09T02:48:17.175888+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.174819+0000","last_change":"2021-03-09T02:48:17.173591+0000","last_active":"2021-03-09T02:48:17.174819+0000","last_peered":"2021-03-09T02:48:17.174819+0000","last_clean":"2021-03-09T02:48:17.174819+0000","last_became_active":"2021-03-09T02:48:17.171711+0000","last_became_peered":"2021-03-09T02:48:17.171711+0000","last_unstale":"2021-03-09T02:48:17.174819+0000","last_undegraded":"2021-03-09T02:48:17.174819+0000","last_fullsized":"2021-03-09T02:48:17.174819+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.174832+0000","last_change":"2021-03-09T02:48:17.173657+0000","last_active":"2021-03-09T02:48:17.174832+0000","last_peered":"2021-03-09T02:48:17.174832+0000","last_clean":"2021-03-09T02:48:17.174832+0000","last_became_active":"2021-03-09T02:48:17.172094+0000","last_became_peered":"2021-03-09T02:48:17.172094+0000","last_unstale":"2021-03-09T02:48:17.174832+0000","last_undegraded":"2021-03-09T02:48:17.174832+0000","last_fullsized":"2021-03-09T02:48:17.174832+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.175855+0000","last_change":"2021-03-09T02:48:17.174318+0000","last_active":"2021-03-09T02:48:17.175855+0000","last_peered":"2021-03-09T02:48:17.175855+0000","last_clean":"2021-03-09T02:48:17.175855+0000","last_became_active":"2021-03-09T02:48:17.173974+0000","last_became_peered":"2021-03-09T02:48:17.173974+0000","last_unstale":"2021-03-09T02:48:17.175855+0000","last_undegraded":"2021-03-09T02:48:17.175855+0000","last_fullsized":"2021-03-09T02:48:17.175855+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.173259+0000","last_change":"2021-03-09T02:48:17.170887+0000","last_active":"2021-03-09T02:48:17.173259+0000","last_peered":"2021-03-09T02:48:17.173259+0000","last_clean":"2021-03-09T02:48:17.173259+0000","last_became_active":"2021-03-09T02:48:17.170519+0000","last_became_peered":"2021-03-09T02:48:17.170519+0000","last_unstale":"2021-03-09T02:48:17.173259+0000","last_undegraded":"2021-03-09T02:48:17.173259+0000","last_fullsized":"2021-03-09T02:48:17.173259+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.174484+0000","last_change":"2021-03-09T02:48:17.171651+0000","last_active":"2021-03-09T02:48:17.174484+0000","last_peered":"2021-03-09T02:48:17.174484+0000","last_clean":"2021-03-09T02:48:17.174484+0000","last_became_active":"2021-03-09T02:48:17.171287+0000","last_became_peered":"2021-03-09T02:48:17.171287+0000","last_unstale":"2021-03-09T02:48:17.174484+0000","last_undegraded":"2021-03-09T02:48:17.174484+0000","last_fullsized":"2021-03-09T02:48:17.174484+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4840,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367000,"statfs":{"total":96636764160,"available":96631808000,"internally_reserved":0,"allocated":98304,"data_stored":26708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2021-03-09T02:48:22.231 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 pg dump --format=json 2021-03-09T02:48:22.524 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:22.526 INFO:teuthology.orchestra.run.smithi092.stderr:dumped all 2021-03-09T02:48:22.538 INFO:teuthology.orchestra.run.smithi092.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-03-09T02:48:21.841412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14648,"kb_used_data":288,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100872,"statfs":{"total":289910292480,"available":289895292928,"internally_reserved":0,"allocated":294912,"data_stored":80124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.174649+0000","last_change":"2021-03-09T02:48:17.173198+0000","last_active":"2021-03-09T02:48:17.174649+0000","last_peered":"2021-03-09T02:48:17.174649+0000","last_clean":"2021-03-09T02:48:17.174649+0000","last_became_active":"2021-03-09T02:48:17.172725+0000","last_became_peered":"2021-03-09T02:48:17.172725+0000","last_unstale":"2021-03-09T02:48:17.174649+0000","last_undegraded":"2021-03-09T02:48:17.174649+0000","last_fullsized":"2021-03-09T02:48:17.174649+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.177042+0000","last_change":"2021-03-09T02:48:17.175331+0000","last_active":"2021-03-09T02:48:17.177042+0000","last_peered":"2021-03-09T02:48:17.177042+0000","last_clean":"2021-03-09T02:48:17.177042+0000","last_became_active":"2021-03-09T02:48:17.174657+0000","last_became_peered":"2021-03-09T02:48:17.174657+0000","last_unstale":"2021-03-09T02:48:17.177042+0000","last_undegraded":"2021-03-09T02:48:17.177042+0000","last_fullsized":"2021-03-09T02:48:17.177042+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.174578+0000","last_change":"2021-03-09T02:48:17.173195+0000","last_active":"2021-03-09T02:48:17.174578+0000","last_peered":"2021-03-09T02:48:17.174578+0000","last_clean":"2021-03-09T02:48:17.174578+0000","last_became_active":"2021-03-09T02:48:17.171539+0000","last_became_peered":"2021-03-09T02:48:17.171539+0000","last_unstale":"2021-03-09T02:48:17.174578+0000","last_undegraded":"2021-03-09T02:48:17.174578+0000","last_fullsized":"2021-03-09T02:48:17.174578+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.175888+0000","last_change":"2021-03-09T02:48:17.174593+0000","last_active":"2021-03-09T02:48:17.175888+0000","last_peered":"2021-03-09T02:48:17.175888+0000","last_clean":"2021-03-09T02:48:17.175888+0000","last_became_active":"2021-03-09T02:48:17.174205+0000","last_became_peered":"2021-03-09T02:48:17.174205+0000","last_unstale":"2021-03-09T02:48:17.175888+0000","last_undegraded":"2021-03-09T02:48:17.175888+0000","last_fullsized":"2021-03-09T02:48:17.175888+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.174819+0000","last_change":"2021-03-09T02:48:17.173591+0000","last_active":"2021-03-09T02:48:17.174819+0000","last_peered":"2021-03-09T02:48:17.174819+0000","last_clean":"2021-03-09T02:48:17.174819+0000","last_became_active":"2021-03-09T02:48:17.171711+0000","last_became_peered":"2021-03-09T02:48:17.171711+0000","last_unstale":"2021-03-09T02:48:17.174819+0000","last_undegraded":"2021-03-09T02:48:17.174819+0000","last_fullsized":"2021-03-09T02:48:17.174819+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.174832+0000","last_change":"2021-03-09T02:48:17.173657+0000","last_active":"2021-03-09T02:48:17.174832+0000","last_peered":"2021-03-09T02:48:17.174832+0000","last_clean":"2021-03-09T02:48:17.174832+0000","last_became_active":"2021-03-09T02:48:17.172094+0000","last_became_peered":"2021-03-09T02:48:17.172094+0000","last_unstale":"2021-03-09T02:48:17.174832+0000","last_undegraded":"2021-03-09T02:48:17.174832+0000","last_fullsized":"2021-03-09T02:48:17.174832+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.175855+0000","last_change":"2021-03-09T02:48:17.174318+0000","last_active":"2021-03-09T02:48:17.175855+0000","last_peered":"2021-03-09T02:48:17.175855+0000","last_clean":"2021-03-09T02:48:17.175855+0000","last_became_active":"2021-03-09T02:48:17.173974+0000","last_became_peered":"2021-03-09T02:48:17.173974+0000","last_unstale":"2021-03-09T02:48:17.175855+0000","last_undegraded":"2021-03-09T02:48:17.175855+0000","last_fullsized":"2021-03-09T02:48:17.175855+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.173259+0000","last_change":"2021-03-09T02:48:17.170887+0000","last_active":"2021-03-09T02:48:17.173259+0000","last_peered":"2021-03-09T02:48:17.173259+0000","last_clean":"2021-03-09T02:48:17.173259+0000","last_became_active":"2021-03-09T02:48:17.170519+0000","last_became_peered":"2021-03-09T02:48:17.170519+0000","last_unstale":"2021-03-09T02:48:17.173259+0000","last_undegraded":"2021-03-09T02:48:17.173259+0000","last_fullsized":"2021-03-09T02:48:17.173259+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:17.174484+0000","last_change":"2021-03-09T02:48:17.171651+0000","last_active":"2021-03-09T02:48:17.174484+0000","last_peered":"2021-03-09T02:48:17.174484+0000","last_clean":"2021-03-09T02:48:17.174484+0000","last_became_active":"2021-03-09T02:48:17.171287+0000","last_became_peered":"2021-03-09T02:48:17.171287+0000","last_unstale":"2021-03-09T02:48:17.174484+0000","last_undegraded":"2021-03-09T02:48:17.174484+0000","last_fullsized":"2021-03-09T02:48:17.174484+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:16.137527+0000","last_clean_scrub_stamp":"2021-03-09T02:48:16.137527+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4840,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367000,"statfs":{"total":96636764160,"available":96631808000,"internally_reserved":0,"allocated":98304,"data_stored":26708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4904,"kb_used_data":96,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366936,"statfs":{"total":96636764160,"available":96631742464,"internally_reserved":0,"allocated":98304,"data_stored":26708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2021-03-09T02:48:22.539 INFO:tasks.ceph.ceph_manager.cluster1:clean! 2021-03-09T02:48:22.539 INFO:tasks.ceph:Waiting until ceph cluster cluster1 is healthy... 2021-03-09T02:48:22.540 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy 2021-03-09T02:48:22.540 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster1 health --format=json 2021-03-09T02:48:22.887 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:22.900 INFO:teuthology.orchestra.run.smithi092.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-03-09T02:48:22.901 INFO:tasks.ceph.ceph_manager.cluster1:wait_until_healthy done 2021-03-09T02:48:22.901 INFO:teuthology.run_tasks:Running task ceph... 2021-03-09T02:48:22.913 INFO:tasks.ceph:Creating ceph cluster cluster2... 2021-03-09T02:48:22.913 INFO:tasks.ceph:config {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'cluster2', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2021-03-09T02:48:22.914 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2021-03-08_21:10:20-rbd-wip-yuri5-testing-2021-03-08-1049-pacific-distro-basic-smithi/5947743', 'branch': 'wip-yuri5-testing-2021-03-08-1049-pacific', 'description': 'rbd/mirror-thrash/{base/install clients/mirror cluster/{2-node openstack} msgr-failures/few objectstore/bluestore-bitmap policy/simple rbd-mirror/four-per-cluster supported-random-distro$/{ubuntu_latest} workloads/rbd-mirror-snapshot-stress-workunit-exclusive-lock}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '5947743', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'meta': [{'desc': 'run two ceph clusters and install rbd-mirror'}, {'desc': 'configure the permissions for client.mirror'}, {'desc': '2 ceph clusters with 1 mon and 3 osds each'}, {'desc': 'run four rbd-mirror daemons per cluster'}, {'desc': 'run the rbd_mirror_stress.sh workunit to test the rbd-mirror daemon'}], 'name': 'yuriw-2021-03-08_21:10:20-rbd-wip-yuri5-testing-2021-03-08-1049-pacific-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri5-testing-2021-03-08-1049-pacific'}, 'ceph': {'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '9e448db9e385343da144613d840349182443aeee'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'sha1': '9e448db9e385343da144613d840349182443aeee'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri5-testing-2021-03-08-1049-pacific', 'sha1': '9e448db9e385343da144613d840349182443aeee'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 80, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['cluster1.mon.a', 'cluster1.mgr.x', 'cluster2.mgr.x', 'cluster1.osd.0', 'cluster1.osd.1', 'cluster1.osd.2', 'cluster1.client.0', 'cluster2.client.0'], ['cluster2.mon.a', 'cluster2.osd.0', 'cluster2.osd.1', 'cluster2.osd.2', 'cluster1.client.mirror', 'cluster1.client.mirror.0', 'cluster1.client.mirror.1', 'cluster1.client.mirror.2', 'cluster1.client.mirror.3', 'cluster1.client.mirror.4', 'cluster1.client.mirror.5', 'cluster1.client.mirror.6', 'cluster2.client.mirror', 'cluster2.client.mirror.0', 'cluster2.client.mirror.1', 'cluster2.client.mirror.2', 'cluster2.client.mirror.3', 'cluster2.client.mirror.4', 'cluster2.client.mirror.5', 'cluster2.client.mirror.6']], 'sha1': '9e448db9e385343da144613d840349182443aeee', 'sleep_before_teardown': 0, 'suite': 'rbd', 'suite_branch': 'wip-yuri5-testing-2021-03-08-1049-pacific', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_9e448db9e385343da144613d840349182443aeee/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '9e448db9e385343da144613d840349182443aeee', 'targets': {'smithi092.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDGM9Wq+tpjME44vX4r5HVAYDKQHOPpKMuKkIN+sN4bXOXrL38m4IlDpX7+u11H/YKmLYO9J8qq8LeIhaPVveNNtI00S6NRcL2gCFTKBc3cGAuajI1w7T5UcmQcjTDdaC6irQDmclSzc/LrhabDa/E8LWZa/J4pJf2j93rHf5ALtl6DxhOPtDMtksmxnIlvXWznZj0nTqZXZjx2V3DJ8edC0ENLQTQuG/GIUX4uAXK5T40vzv21g1EG4jAiUKZXB1t91+utaws6miPeVZ6W94mAkMdmqdlkcbccy4PWRXiz4+MNFNhUtNBxtSgeuFc8gQfrZXKWdIxyHpSoybONFFN/', 'smithi144.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQClEsHrqwB5WWdjBzHc5HfVxIvP9a9tTOK/FXO13yldAlBA6wEnj8ybBtfnvsWbhjdORYKBxUEiPAcNjeDd6CdsyLAIAI/Yo6n//ZuBnBEURpcBxLmeyQf8YLvKLJ8eUYJTdiFMUiimvqszh7S1Uzqi5crdAP9jWyZjL7Bo+7itY52pzDWhxFtARQfcV/FOUBQymt8GK7We3s1Dlc/vfHw0JTaxeUfwCBOx2L2GxeGAGLpnnJhV6ohsFhIb7ZnfMRAR2PV4osf84S67vArQj7H5H/XqQJaSVc5EUkPodGwiRxqplJCgtlpb205SiEMOBwB/XX5ppSSCp0RfBj91NwdX'}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': ['rbd-mirror'], 'sha1': '9e448db9e385343da144613d840349182443aeee'}}, {'ceph': {'cluster': 'cluster1', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '9e448db9e385343da144613d840349182443aeee'}}, {'ceph': {'cluster': 'cluster2', 'conf': {'client': {'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'global': {'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '9e448db9e385343da144613d840349182443aeee'}}, {'exec': {'cluster1.client.mirror.0': ["sudo ceph --cluster cluster1 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster1 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror mon 'profile rbd-mirror-peer' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.0 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.1 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.2 mon 'profile rbd-mirror' osd 'profile rbd'", "sudo ceph --cluster cluster2 auth caps client.mirror.3 mon 'profile rbd-mirror' osd 'profile rbd'"]}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster1.client.mirror.3', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.0', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.1', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.2', 'thrash': True}}, {'rbd-mirror': {'client': 'cluster2.client.mirror.3', 'thrash': True}}, {'rbd-mirror-thrash': {'cluster': 'cluster1'}}, {'rbd-mirror-thrash': {'cluster': 'cluster2'}}, {'workunit': {'clients': {'cluster1.client.mirror': ['rbd/rbd_mirror_stress.sh']}, 'env': {'CEPH_ARGS': '', 'MIRROR_IMAGE_MODE': 'snapshot', 'MIRROR_POOL_MODE': 'image', 'RBD_IMAGE_FEATURES': 'layering,exclusive-lock', 'RBD_MIRROR_INSTANCES': '4', 'RBD_MIRROR_USE_EXISTING_CLUSTER': '1', 'RBD_MIRROR_USE_RBD_MIRROR': '1'}, 'timeout': '6h'}}], 'teuthology_branch': 'master', 'teuthology_sha1': '3082387bba74fcd24c9700593d10418152d53c97', 'timestamp': '2021-03-08_21:10:20', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.25973'} 2021-03-09T02:48:22.915 DEBUG:teuthology.orchestra.run.smithi092:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2021-03-09T02:48:22.917 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/cluster2.data 2021-03-09T02:48:22.926 DEBUG:teuthology.orchestra.run.smithi092:> sudo install -d -m0777 -- /var/run/ceph 2021-03-09T02:48:22.965 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m0777 -- /var/run/ceph 2021-03-09T02:48:22.982 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:22.982 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2021-03-09T02:48:23.029 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-03-09T02:48:23.030 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-03-09T02:48:23.077 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-03-09T02:48:23.078 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-09T02:48:23.078 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 340 Links: 1 2021-03-09T02:48:23.078 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-09T02:48:23.079 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2021-03-09 02:45:18.883467558 +0000 2021-03-09T02:48:23.079 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2021-03-09 02:45:18.779467561 +0000 2021-03-09T02:48:23.079 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2021-03-09 02:45:18.779467561 +0000 2021-03-09T02:48:23.079 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2021-03-09T02:48:23.087 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2021-03-09T02:48:23.087 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2021-03-09T02:48:23.088 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000395564 s, 1.3 MB/s 2021-03-09T02:48:23.092 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-03-09T02:48:23.146 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-03-09T02:48:23.147 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-09T02:48:23.147 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 347 Links: 1 2021-03-09T02:48:23.147 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-09T02:48:23.148 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2021-03-09 02:45:19.139467551 +0000 2021-03-09T02:48:23.149 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2021-03-09 02:45:19.023467555 +0000 2021-03-09T02:48:23.149 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2021-03-09 02:45:19.023467555 +0000 2021-03-09T02:48:23.150 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2021-03-09T02:48:23.155 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2021-03-09T02:48:23.156 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2021-03-09T02:48:23.156 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000323526 s, 1.6 MB/s 2021-03-09T02:48:23.160 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-03-09T02:48:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-03-09T02:48:23.210 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-09T02:48:23.210 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 353 Links: 1 2021-03-09T02:48:23.210 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-09T02:48:23.211 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2021-03-09 02:45:19.279467547 +0000 2021-03-09T02:48:23.211 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2021-03-09 02:45:19.275467548 +0000 2021-03-09T02:48:23.211 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2021-03-09 02:45:19.275467548 +0000 2021-03-09T02:48:23.211 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2021-03-09T02:48:23.219 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2021-03-09T02:48:23.219 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2021-03-09T02:48:23.220 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000324194 s, 1.6 MB/s 2021-03-09T02:48:23.223 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-03-09T02:48:23.274 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-03-09T02:48:23.274 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-03-09T02:48:23.274 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 6h/6d Inode: 359 Links: 1 2021-03-09T02:48:23.274 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-03-09T02:48:23.275 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2021-03-09 02:45:19.547467540 +0000 2021-03-09T02:48:23.275 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2021-03-09 02:45:19.543467540 +0000 2021-03-09T02:48:23.275 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2021-03-09 02:45:19.543467540 +0000 2021-03-09T02:48:23.276 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2021-03-09T02:48:23.283 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2021-03-09T02:48:23.283 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2021-03-09T02:48:23.284 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000417008 s, 1.2 MB/s 2021-03-09T02:48:23.288 INFO:tasks.ceph:osd dev map: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-09T02:48:23.288 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2021-03-09T02:48:23.289 INFO:tasks.ceph:Generating config... 2021-03-09T02:48:23.290 INFO:tasks.ceph:[client] debug rbd = 20 2021-03-09T02:48:23.290 INFO:tasks.ceph:[client] debug rbd_mirror = 15 2021-03-09T02:48:23.291 INFO:tasks.ceph:[client] log to stderr = False 2021-03-09T02:48:23.291 INFO:tasks.ceph:[client] rbd default features = 125 2021-03-09T02:48:23.291 INFO:tasks.ceph:[client] rbd mirror image policy type = simple 2021-03-09T02:48:23.292 INFO:tasks.ceph:[client.mirror.0] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.292 INFO:tasks.ceph:[client.mirror.0] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.292 INFO:tasks.ceph:[client.mirror.1] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.293 INFO:tasks.ceph:[client.mirror.1] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.293 INFO:tasks.ceph:[client.mirror.2] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.294 INFO:tasks.ceph:[client.mirror.2] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.294 INFO:tasks.ceph:[client.mirror.3] admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.295 INFO:tasks.ceph:[client.mirror.3] pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.295 INFO:tasks.ceph:[global] mon client directed command retry = 5 2021-03-09T02:48:23.298 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2021-03-09T02:48:23.299 INFO:tasks.ceph:[mgr] debug mgr = 20 2021-03-09T02:48:23.299 INFO:tasks.ceph:[mgr] debug ms = 1 2021-03-09T02:48:23.300 INFO:tasks.ceph:[mon] debug mon = 20 2021-03-09T02:48:23.300 INFO:tasks.ceph:[mon] debug ms = 1 2021-03-09T02:48:23.301 INFO:tasks.ceph:[mon] debug paxos = 20 2021-03-09T02:48:23.301 INFO:tasks.ceph:[osd] bdev async discard = True 2021-03-09T02:48:23.302 INFO:tasks.ceph:[osd] bdev enable discard = True 2021-03-09T02:48:23.302 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2021-03-09T02:48:23.302 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2021-03-09T02:48:23.303 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2021-03-09T02:48:23.303 INFO:tasks.ceph:[osd] debug bluefs = 20 2021-03-09T02:48:23.303 INFO:tasks.ceph:[osd] debug bluestore = 20 2021-03-09T02:48:23.304 INFO:tasks.ceph:[osd] debug filestore = 20 2021-03-09T02:48:23.304 INFO:tasks.ceph:[osd] debug journal = 20 2021-03-09T02:48:23.305 INFO:tasks.ceph:[osd] debug ms = 1 2021-03-09T02:48:23.305 INFO:tasks.ceph:[osd] debug osd = 25 2021-03-09T02:48:23.305 INFO:tasks.ceph:[osd] debug rocksdb = 10 2021-03-09T02:48:23.306 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2021-03-09T02:48:23.307 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2021-03-09T02:48:23.308 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2021-03-09T02:48:23.308 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2021-03-09T02:48:23.308 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2021-03-09T02:48:23.309 INFO:tasks.ceph:Setting up cluster2.mon.a... 2021-03-09T02:48:23.309 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/cluster2.keyring 2021-03-09T02:48:23.363 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster2.keyring 2021-03-09T02:48:23.367 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/cluster2.keyring 2021-03-09T02:48:23.446 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-03-09T02:48:23.501 DEBUG:tasks.ceph:Ceph mon addresses: [('cluster2.mon.a', '172.21.15.144')] 2021-03-09T02:48:23.502 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.144', 'mon client directed command retry': 5, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 1, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug rbd': 20, 'debug rbd_mirror': 15, 'log to stderr': False, 'rbd default features': 125, 'rbd mirror image policy type': 'simple'}, 'mon.a': {}, 'client.mirror.0': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.1': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.2': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}, 'client.mirror.3': {'admin socket': '/var/run/ceph/rbd-mirror.$cluster-$name.asok', 'pid file': '/var/run/ceph/rbd-mirror.$cluster-$name.pid'}} 2021-03-09T02:48:23.503 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:23.504 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2021-03-09T02:48:23.549 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.144 --print /home/ubuntu/cephtest/cluster2.monmap 2021-03-09T02:48:23.621 INFO:teuthology.orchestra.run.smithi144.stdout:monmaptool: monmap file /home/ubuntu/cephtest/cluster2.monmap 2021-03-09T02:48:23.622 INFO:teuthology.orchestra.run.smithi144.stdout:monmaptool: generated fsid f298a705-d51f-4162-ac79-39dab9e11f91 2021-03-09T02:48:23.622 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 0 2021-03-09T02:48:23.622 INFO:teuthology.orchestra.run.smithi144.stdout:fsid f298a705-d51f-4162-ac79-39dab9e11f91 2021-03-09T02:48:23.623 INFO:teuthology.orchestra.run.smithi144.stdout:last_changed 2021-03-09T02:48:23.622924+0000 2021-03-09T02:48:23.623 INFO:teuthology.orchestra.run.smithi144.stdout:created 2021-03-09T02:48:23.622924+0000 2021-03-09T02:48:23.623 INFO:teuthology.orchestra.run.smithi144.stdout:min_mon_release 0 (unknown) 2021-03-09T02:48:23.623 INFO:teuthology.orchestra.run.smithi144.stdout:election_strategy: 1 2021-03-09T02:48:23.623 INFO:teuthology.orchestra.run.smithi144.stdout:0: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.a 2021-03-09T02:48:23.624 INFO:teuthology.orchestra.run.smithi144.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/cluster2.monmap (1 monitors) 2021-03-09T02:48:23.625 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2021-03-09T02:48:23.673 INFO:tasks.ceph:Writing /etc/ceph/cluster2.conf for FSID f298a705-d51f-4162-ac79-39dab9e11f91... 2021-03-09T02:48:23.674 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2021-03-09T02:48:23.677 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/cluster2.conf && sudo chmod 0644 /etc/ceph/cluster2.conf > /dev/null 2021-03-09T02:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout:[global] 2021-03-09T02:48:23.718 INFO:teuthology.orchestra.run.smithi092.stdout: chdir = "" 2021-03-09T02:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-03-09T02:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: auth supported = cephx 2021-03-09T02:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: filestore xattr use omap = true 2021-03-09T02:48:23.719 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: mon clock drift allowed = 1.000 2021-03-09T02:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: osd crush chooseleaf type = 0 2021-03-09T02:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: auth debug = true 2021-03-09T02:48:23.720 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on old message = true 2021-03-09T02:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: ms die on bug = true 2021-03-09T02:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.721 INFO:teuthology.orchestra.run.smithi092.stdout: mon max pg per osd = 10000 # >= luminous 2021-03-09T02:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: mon pg warn max object skew = 0 2021-03-09T02:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: # disable pg_autoscaler by default for new pools 2021-03-09T02:48:23.722 INFO:teuthology.orchestra.run.smithi092.stdout: osd_pool_default_pg_autoscale_mode = off 2021-03-09T02:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default size = 2 2021-03-09T02:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.723 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow primary affinity = true 2021-03-09T02:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd allow pg remap = true 2021-03-09T02:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on legacy crush tunables = false 2021-03-09T02:48:23.724 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on crush straw calc version zero = false 2021-03-09T02:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on no sortbitwise = false 2021-03-09T02:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on osd down out interval zero = false 2021-03-09T02:48:23.725 INFO:teuthology.orchestra.run.smithi092.stdout: mon warn on too few osds = false 2021-03-09T02:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-03-09T02:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: mon_warn_on_pool_no_redundancy = false 2021-03-09T02:48:23.726 INFO:teuthology.orchestra.run.smithi092.stdout: mon_allow_pool_size_one = true 2021-03-09T02:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-03-09T02:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.727 INFO:teuthology.orchestra.run.smithi092.stdout: osd default data pool replay window = 5 2021-03-09T02:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: mon allow pool delete = true 2021-03-09T02:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: mon cluster log file level = debug 2021-03-09T02:48:23.728 INFO:teuthology.orchestra.run.smithi092.stdout: debug asserts on shutdown = true 2021-03-09T02:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: mon health detail to clog = false 2021-03-09T02:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: mon host = 172.21.15.144 2021-03-09T02:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: mon client directed command retry = 5 2021-03-09T02:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: ms inject socket failures = 5000 2021-03-09T02:48:23.729 INFO:teuthology.orchestra.run.smithi092.stdout: fsid = f298a705-d51f-4162-ac79-39dab9e11f91 2021-03-09T02:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout:[osd] 2021-03-09T02:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: osd journal size = 100 2021-03-09T02:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.730 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub load threshold = 5.0 2021-03-09T02:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: osd scrub max interval = 600 2021-03-09T02:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: osd recover clone overlap = true 2021-03-09T02:48:23.731 INFO:teuthology.orchestra.run.smithi092.stdout: osd recovery max chunk = 1048576 2021-03-09T02:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug shutdown = true 2021-03-09T02:48:23.732 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug op order = true 2021-03-09T02:48:23.733 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug verify stray on activate = true 2021-03-09T02:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: osd open classes on start = true 2021-03-09T02:48:23.734 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug pg log writeout = true 2021-03-09T02:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: osd deep scrub update digest min age = 30 2021-03-09T02:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.735 INFO:teuthology.orchestra.run.smithi092.stdout: osd map max advance = 10 2021-03-09T02:48:23.736 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.736 INFO:teuthology.orchestra.run.smithi092.stdout: journal zero on create = true 2021-03-09T02:48:23.736 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.736 INFO:teuthology.orchestra.run.smithi092.stdout: filestore ondisk finisher threads = 3 2021-03-09T02:48:23.737 INFO:teuthology.orchestra.run.smithi092.stdout: filestore apply finisher threads = 3 2021-03-09T02:48:23.737 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.737 INFO:teuthology.orchestra.run.smithi092.stdout: bdev debug aio = true 2021-03-09T02:48:23.737 INFO:teuthology.orchestra.run.smithi092.stdout: osd debug misdirected ops = true 2021-03-09T02:48:23.738 INFO:teuthology.orchestra.run.smithi092.stdout: bdev async discard = True 2021-03-09T02:48:23.738 INFO:teuthology.orchestra.run.smithi092.stdout: bdev enable discard = True 2021-03-09T02:48:23.738 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore allocator = bitmap 2021-03-09T02:48:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore block size = 96636764160 2021-03-09T02:48:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: bluestore fsck on mount = True 2021-03-09T02:48:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluefs = 20 2021-03-09T02:48:23.739 INFO:teuthology.orchestra.run.smithi092.stdout: debug bluestore = 20 2021-03-09T02:48:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: debug filestore = 20 2021-03-09T02:48:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: debug journal = 20 2021-03-09T02:48:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2021-03-09T02:48:23.740 INFO:teuthology.orchestra.run.smithi092.stdout: debug osd = 25 2021-03-09T02:48:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: debug rocksdb = 10 2021-03-09T02:48:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd backfillfull_ratio = 0.85 2021-03-09T02:48:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd full ratio = 0.9 2021-03-09T02:48:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd nearfull ratio = 0.8 2021-03-09T02:48:23.741 INFO:teuthology.orchestra.run.smithi092.stdout: osd failsafe full ratio = 0.95 2021-03-09T02:48:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: osd objectstore = bluestore 2021-03-09T02:48:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.742 INFO:teuthology.orchestra.run.smithi092.stdout:[mgr] 2021-03-09T02:48:23.742 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2021-03-09T02:48:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: debug mgr = 20 2021-03-09T02:48:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2021-03-09T02:48:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2021-03-09T02:48:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2021-03-09T02:48:23.743 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2021-03-09T02:48:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: mgr/telemetry/nag = false 2021-03-09T02:48:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.744 INFO:teuthology.orchestra.run.smithi092.stdout:[mon] 2021-03-09T02:48:23.744 INFO:teuthology.orchestra.run.smithi092.stdout: debug ms = 1 2021-03-09T02:48:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: debug mon = 20 2021-03-09T02:48:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: debug paxos = 20 2021-03-09T02:48:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: debug auth = 20 2021-03-09T02:48:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: mon data avail warn = 5 2021-03-09T02:48:23.745 INFO:teuthology.orchestra.run.smithi092.stdout: mon mgr mkfs grace = 240 2021-03-09T02:48:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min pgs per osd = 4 2021-03-09T02:48:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd reporter subtree level = osd 2021-03-09T02:48:23.746 INFO:teuthology.orchestra.run.smithi092.stdout: mon osd prime pg temp = true 2021-03-09T02:48:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: mon reweight min bytes per osd = 10 2021-03-09T02:48:23.747 INFO:teuthology.orchestra.run.smithi092.stdout: 2021-03-09T02:48:23.747 INFO:teuthology.orchestra.run.smithi092.stdout:[client] 2021-03-09T02:48:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: rgw cache enabled = true 2021-03-09T02:48:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable ops log = true 2021-03-09T02:48:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: rgw enable usage log = true 2021-03-09T02:48:23.748 INFO:teuthology.orchestra.run.smithi092.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-03-09T02:48:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-03-09T02:48:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: debug rbd = 20 2021-03-09T02:48:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: debug rbd_mirror = 15 2021-03-09T02:48:23.749 INFO:teuthology.orchestra.run.smithi092.stdout: log to stderr = False 2021-03-09T02:48:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: rbd default features = 125 2021-03-09T02:48:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: rbd mirror image policy type = simple 2021-03-09T02:48:23.750 INFO:teuthology.orchestra.run.smithi092.stdout:[mon.a] 2021-03-09T02:48:23.750 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.0] 2021-03-09T02:48:23.750 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.751 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.1] 2021-03-09T02:48:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.751 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.752 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.2] 2021-03-09T02:48:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.752 INFO:teuthology.orchestra.run.smithi092.stdout:[client.mirror.3] 2021-03-09T02:48:23.752 INFO:teuthology.orchestra.run.smithi092.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.753 INFO:teuthology.orchestra.run.smithi092.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.754 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2021-03-09T02:48:23.754 INFO:teuthology.orchestra.run.smithi144.stdout: chdir = "" 2021-03-09T02:48:23.754 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2021-03-09T02:48:23.754 INFO:teuthology.orchestra.run.smithi144.stdout: auth supported = cephx 2021-03-09T02:48:23.755 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.755 INFO:teuthology.orchestra.run.smithi144.stdout: filestore xattr use omap = true 2021-03-09T02:48:23.755 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.756 INFO:teuthology.orchestra.run.smithi144.stdout: mon clock drift allowed = 1.000 2021-03-09T02:48:23.756 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.756 INFO:teuthology.orchestra.run.smithi144.stdout: osd crush chooseleaf type = 0 2021-03-09T02:48:23.756 INFO:teuthology.orchestra.run.smithi144.stdout: auth debug = true 2021-03-09T02:48:23.757 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.757 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on old message = true 2021-03-09T02:48:23.757 INFO:teuthology.orchestra.run.smithi144.stdout: ms die on bug = true 2021-03-09T02:48:23.758 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.758 INFO:teuthology.orchestra.run.smithi144.stdout: mon max pg per osd = 10000 # >= luminous 2021-03-09T02:48:23.758 INFO:teuthology.orchestra.run.smithi144.stdout: mon pg warn max object skew = 0 2021-03-09T02:48:23.759 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.759 INFO:teuthology.orchestra.run.smithi144.stdout: # disable pg_autoscaler by default for new pools 2021-03-09T02:48:23.759 INFO:teuthology.orchestra.run.smithi144.stdout: osd_pool_default_pg_autoscale_mode = off 2021-03-09T02:48:23.759 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.759 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default size = 2 2021-03-09T02:48:23.760 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.760 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow primary affinity = true 2021-03-09T02:48:23.760 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd allow pg remap = true 2021-03-09T02:48:23.760 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on legacy crush tunables = false 2021-03-09T02:48:23.761 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on crush straw calc version zero = false 2021-03-09T02:48:23.761 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on no sortbitwise = false 2021-03-09T02:48:23.761 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on osd down out interval zero = false 2021-03-09T02:48:23.761 INFO:teuthology.orchestra.run.smithi144.stdout: mon warn on too few osds = false 2021-03-09T02:48:23.762 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2021-03-09T02:48:23.762 INFO:teuthology.orchestra.run.smithi144.stdout: mon_warn_on_pool_no_redundancy = false 2021-03-09T02:48:23.762 INFO:teuthology.orchestra.run.smithi144.stdout: mon_allow_pool_size_one = true 2021-03-09T02:48:23.762 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.763 INFO:teuthology.orchestra.run.smithi144.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2021-03-09T02:48:23.763 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.763 INFO:teuthology.orchestra.run.smithi144.stdout: osd default data pool replay window = 5 2021-03-09T02:48:23.763 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.763 INFO:teuthology.orchestra.run.smithi144.stdout: mon allow pool delete = true 2021-03-09T02:48:23.764 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.764 INFO:teuthology.orchestra.run.smithi144.stdout: mon cluster log file level = debug 2021-03-09T02:48:23.764 INFO:teuthology.orchestra.run.smithi144.stdout: debug asserts on shutdown = true 2021-03-09T02:48:23.764 INFO:teuthology.orchestra.run.smithi144.stdout: mon health detail to clog = false 2021-03-09T02:48:23.765 INFO:teuthology.orchestra.run.smithi144.stdout: mon host = 172.21.15.144 2021-03-09T02:48:23.765 INFO:teuthology.orchestra.run.smithi144.stdout: mon client directed command retry = 5 2021-03-09T02:48:23.765 INFO:teuthology.orchestra.run.smithi144.stdout: ms inject socket failures = 5000 2021-03-09T02:48:23.765 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = f298a705-d51f-4162-ac79-39dab9e11f91 2021-03-09T02:48:23.766 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.766 INFO:teuthology.orchestra.run.smithi144.stdout:[osd] 2021-03-09T02:48:23.766 INFO:teuthology.orchestra.run.smithi144.stdout: osd journal size = 100 2021-03-09T02:48:23.767 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.767 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub load threshold = 5.0 2021-03-09T02:48:23.767 INFO:teuthology.orchestra.run.smithi144.stdout: osd scrub max interval = 600 2021-03-09T02:48:23.767 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.768 INFO:teuthology.orchestra.run.smithi144.stdout: osd recover clone overlap = true 2021-03-09T02:48:23.768 INFO:teuthology.orchestra.run.smithi144.stdout: osd recovery max chunk = 1048576 2021-03-09T02:48:23.768 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.768 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug shutdown = true 2021-03-09T02:48:23.769 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug op order = true 2021-03-09T02:48:23.769 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug verify stray on activate = true 2021-03-09T02:48:23.769 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.769 INFO:teuthology.orchestra.run.smithi144.stdout: osd open classes on start = true 2021-03-09T02:48:23.770 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug pg log writeout = true 2021-03-09T02:48:23.770 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.770 INFO:teuthology.orchestra.run.smithi144.stdout: osd deep scrub update digest min age = 30 2021-03-09T02:48:23.770 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.770 INFO:teuthology.orchestra.run.smithi144.stdout: osd map max advance = 10 2021-03-09T02:48:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: journal zero on create = true 2021-03-09T02:48:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.771 INFO:teuthology.orchestra.run.smithi144.stdout: filestore ondisk finisher threads = 3 2021-03-09T02:48:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: filestore apply finisher threads = 3 2021-03-09T02:48:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: bdev debug aio = true 2021-03-09T02:48:23.772 INFO:teuthology.orchestra.run.smithi144.stdout: osd debug misdirected ops = true 2021-03-09T02:48:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: bdev async discard = True 2021-03-09T02:48:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: bdev enable discard = True 2021-03-09T02:48:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore allocator = bitmap 2021-03-09T02:48:23.773 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore block size = 96636764160 2021-03-09T02:48:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: bluestore fsck on mount = True 2021-03-09T02:48:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluefs = 20 2021-03-09T02:48:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: debug bluestore = 20 2021-03-09T02:48:23.774 INFO:teuthology.orchestra.run.smithi144.stdout: debug filestore = 20 2021-03-09T02:48:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: debug journal = 20 2021-03-09T02:48:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2021-03-09T02:48:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: debug osd = 25 2021-03-09T02:48:23.775 INFO:teuthology.orchestra.run.smithi144.stdout: debug rocksdb = 10 2021-03-09T02:48:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd backfillfull_ratio = 0.85 2021-03-09T02:48:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd full ratio = 0.9 2021-03-09T02:48:23.776 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd nearfull ratio = 0.8 2021-03-09T02:48:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: osd failsafe full ratio = 0.95 2021-03-09T02:48:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: osd objectstore = bluestore 2021-03-09T02:48:23.777 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.777 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr] 2021-03-09T02:48:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2021-03-09T02:48:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: debug mgr = 20 2021-03-09T02:48:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2021-03-09T02:48:23.778 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2021-03-09T02:48:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2021-03-09T02:48:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2021-03-09T02:48:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: mgr/telemetry/nag = false 2021-03-09T02:48:23.779 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.780 INFO:teuthology.orchestra.run.smithi144.stdout:[mon] 2021-03-09T02:48:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: debug ms = 1 2021-03-09T02:48:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: debug mon = 20 2021-03-09T02:48:23.780 INFO:teuthology.orchestra.run.smithi144.stdout: debug paxos = 20 2021-03-09T02:48:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: debug auth = 20 2021-03-09T02:48:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: mon data avail warn = 5 2021-03-09T02:48:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: mon mgr mkfs grace = 240 2021-03-09T02:48:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min pgs per osd = 4 2021-03-09T02:48:23.781 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd reporter subtree level = osd 2021-03-09T02:48:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: mon osd prime pg temp = true 2021-03-09T02:48:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: mon reweight min bytes per osd = 10 2021-03-09T02:48:23.782 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:23.782 INFO:teuthology.orchestra.run.smithi144.stdout:[client] 2021-03-09T02:48:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: rgw cache enabled = true 2021-03-09T02:48:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable ops log = true 2021-03-09T02:48:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: rgw enable usage log = true 2021-03-09T02:48:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2021-03-09T02:48:23.783 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2021-03-09T02:48:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: debug rbd = 20 2021-03-09T02:48:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: debug rbd_mirror = 15 2021-03-09T02:48:23.784 INFO:teuthology.orchestra.run.smithi144.stdout: log to stderr = False 2021-03-09T02:48:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: rbd default features = 125 2021-03-09T02:48:23.785 INFO:teuthology.orchestra.run.smithi144.stdout: rbd mirror image policy type = simple 2021-03-09T02:48:23.785 INFO:teuthology.orchestra.run.smithi144.stdout:[mon.a] 2021-03-09T02:48:23.786 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.0] 2021-03-09T02:48:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.786 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.786 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.1] 2021-03-09T02:48:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.787 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.2] 2021-03-09T02:48:23.787 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.788 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.3] 2021-03-09T02:48:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: admin socket = /var/run/ceph/rbd-mirror.$cluster-$name.asok 2021-03-09T02:48:23.788 INFO:teuthology.orchestra.run.smithi144.stdout: pid file = /var/run/ceph/rbd-mirror.$cluster-$name.pid 2021-03-09T02:48:23.791 INFO:tasks.ceph:Creating admin key on cluster2.mon.a... 2021-03-09T02:48:23.791 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/cluster2.keyring 2021-03-09T02:48:23.879 INFO:tasks.ceph:Copying monmap to all nodes... 2021-03-09T02:48:23.879 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:23.879 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster2.keyring of=/dev/stdout 2021-03-09T02:48:23.885 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:23.886 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/home/ubuntu/cephtest/cluster2.monmap of=/dev/stdout 2021-03-09T02:48:23.933 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi092.front.sepia.ceph.com 2021-03-09T02:48:23.934 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:23.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd of=/etc/ceph/cluster2.keyring 2021-03-09T02:48:23.934 DEBUG:teuthology.orchestra.run.smithi092:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-03-09T02:48:23.958 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:23.959 DEBUG:teuthology.orchestra.run.smithi092:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2021-03-09T02:48:24.009 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi144.front.sepia.ceph.com 2021-03-09T02:48:24.010 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:24.010 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/cluster2.keyring 2021-03-09T02:48:24.011 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/cluster2.keyring 2021-03-09T02:48:24.035 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:24.036 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/cluster2.monmap 2021-03-09T02:48:24.085 INFO:tasks.ceph:Setting up mon nodes... 2021-03-09T02:48:24.086 INFO:tasks.ceph:Setting up mgr nodes... 2021-03-09T02:48:24.086 DEBUG:teuthology.orchestra.run.smithi092:> sudo mkdir -p /var/lib/ceph/mgr/cluster2-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/cluster2-x/keyring 2021-03-09T02:48:24.129 INFO:teuthology.orchestra.run.smithi092.stdout:creating /var/lib/ceph/mgr/cluster2-x/keyring 2021-03-09T02:48:24.131 INFO:tasks.ceph:Setting up mds nodes... 2021-03-09T02:48:24.132 INFO:tasks.ceph_client:Setting up client nodes... 2021-03-09T02:48:24.132 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/cluster2.client.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.0.keyring 2021-03-09T02:48:24.207 INFO:teuthology.orchestra.run.smithi092.stdout:creating /etc/ceph/cluster2.client.0.keyring 2021-03-09T02:48:24.218 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror /etc/ceph/cluster2.client.mirror.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.keyring 2021-03-09T02:48:24.252 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster2.client.mirror.keyring 2021-03-09T02:48:24.263 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.0 /etc/ceph/cluster2.client.mirror.0.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.0.keyring 2021-03-09T02:48:24.340 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster2.client.mirror.0.keyring 2021-03-09T02:48:24.352 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.1 /etc/ceph/cluster2.client.mirror.1.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.1.keyring 2021-03-09T02:48:24.429 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster2.client.mirror.1.keyring 2021-03-09T02:48:24.441 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.2 /etc/ceph/cluster2.client.mirror.2.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.2.keyring 2021-03-09T02:48:24.516 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster2.client.mirror.2.keyring 2021-03-09T02:48:24.528 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.3 /etc/ceph/cluster2.client.mirror.3.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.3.keyring 2021-03-09T02:48:24.603 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster2.client.mirror.3.keyring 2021-03-09T02:48:24.616 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.4 /etc/ceph/cluster2.client.mirror.4.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.4.keyring 2021-03-09T02:48:24.692 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster2.client.mirror.4.keyring 2021-03-09T02:48:24.703 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.5 /etc/ceph/cluster2.client.mirror.5.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.5.keyring 2021-03-09T02:48:24.780 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster2.client.mirror.5.keyring 2021-03-09T02:48:24.791 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.mirror.6 /etc/ceph/cluster2.client.mirror.6.keyring && sudo chmod 0644 /etc/ceph/cluster2.client.mirror.6.keyring 2021-03-09T02:48:24.867 INFO:teuthology.orchestra.run.smithi144.stdout:creating /etc/ceph/cluster2.client.mirror.6.keyring 2021-03-09T02:48:24.879 INFO:tasks.ceph:Running mkfs on osd nodes... 2021-03-09T02:48:24.879 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi092.front.sepia.ceph.com'): {'cluster1.osd.0': '/dev/vg_nvme/lv_1', 'cluster1.osd.1': '/dev/vg_nvme/lv_2', 'cluster1.osd.2': '/dev/vg_nvme/lv_3'}, Remote(name='ubuntu@smithi144.front.sepia.ceph.com'): {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'}} 2021-03-09T02:48:24.880 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/cluster2-0 2021-03-09T02:48:24.936 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-09T02:48:24.936 INFO:tasks.ceph:role: cluster2.osd.0 2021-03-09T02:48:24.937 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com 2021-03-09T02:48:24.937 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2021-03-09T02:48:25.454 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2021-03-09T02:48:25.454 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-09T02:48:25.454 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-09T02:48:25.455 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-09T02:48:25.455 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2021-03-09T02:48:25.455 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-09T02:48:25.455 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-09T02:48:25.455 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-09T02:48:25.456 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-09T02:48:25.457 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2021-03-09T02:48:25.457 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/cluster2-0 2021-03-09T02:48:25.539 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-0 2021-03-09T02:48:25.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:48:25.552 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2021-03-09T02:48:25.552 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/cluster2-1 2021-03-09T02:48:25.610 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-09T02:48:25.611 INFO:tasks.ceph:role: cluster2.osd.1 2021-03-09T02:48:25.611 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com 2021-03-09T02:48:25.611 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2021-03-09T02:48:26.115 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2021-03-09T02:48:26.115 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-09T02:48:26.115 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-09T02:48:26.116 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-09T02:48:26.116 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2021-03-09T02:48:26.116 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-09T02:48:26.116 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-09T02:48:26.117 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-09T02:48:26.117 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-09T02:48:26.118 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2021-03-09T02:48:26.118 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/cluster2-1 2021-03-09T02:48:26.202 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-1 2021-03-09T02:48:26.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:48:26.214 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2021-03-09T02:48:26.214 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/osd/cluster2-2 2021-03-09T02:48:26.270 INFO:tasks.ceph:roles_to_devs: {'cluster2.osd.0': '/dev/vg_nvme/lv_1', 'cluster2.osd.1': '/dev/vg_nvme/lv_2', 'cluster2.osd.2': '/dev/vg_nvme/lv_3'} 2021-03-09T02:48:26.270 INFO:tasks.ceph:role: cluster2.osd.2 2021-03-09T02:48:26.271 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com 2021-03-09T02:48:26.271 DEBUG:teuthology.orchestra.run.smithi144:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2021-03-09T02:48:26.768 INFO:teuthology.orchestra.run.smithi144.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2021-03-09T02:48:26.769 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 attr=2, projid32bit=1 2021-03-09T02:48:26.769 INFO:teuthology.orchestra.run.smithi144.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2021-03-09T02:48:26.769 INFO:teuthology.orchestra.run.smithi144.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2021-03-09T02:48:26.769 INFO:teuthology.orchestra.run.smithi144.stdout: = sunit=0 swidth=0 blks 2021-03-09T02:48:26.770 INFO:teuthology.orchestra.run.smithi144.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2021-03-09T02:48:26.770 INFO:teuthology.orchestra.run.smithi144.stdout:log =internal log bsize=4096 blocks=11443, version=2 2021-03-09T02:48:26.770 INFO:teuthology.orchestra.run.smithi144.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2021-03-09T02:48:26.770 INFO:teuthology.orchestra.run.smithi144.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2021-03-09T02:48:26.771 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi144.front.sepia.ceph.com -o noatime 2021-03-09T02:48:26.772 DEBUG:teuthology.orchestra.run.smithi144:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/cluster2-2 2021-03-09T02:48:26.826 DEBUG:teuthology.orchestra.run.smithi144:> sudo /sbin/restorecon /var/lib/ceph/osd/cluster2-2 2021-03-09T02:48:26.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:48:26.838 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: /sbin/restorecon: command not found 2021-03-09T02:48:26.838 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-03-09T02:48:26.937 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:26.936+0000 7f22e5322080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-0/keyring: can't open /var/lib/ceph/osd/cluster2-0/keyring: (2) No such file or directory 2021-03-09T02:48:26.939 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:26.936+0000 7f22e5322080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-0/keyring 2021-03-09T02:48:26.939 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:26.936+0000 7f22e5322080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-03-09T02:48:26.940 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:26.936+0000 7f22e5322080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-03-09T02:48:26.940 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:26.936+0000 7f22e5322080 -1 bluestore(/var/lib/ceph/osd/cluster2-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-0/block: (2) No such file or directory 2021-03-09T02:48:26.942 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:26.940+0000 7f22e5322080 -1 bluestore(/var/lib/ceph/osd/cluster2-0) _read_fsid unparsable uuid 2021-03-09T02:48:29.896 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-0 2021-03-09T02:48:29.912 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-03-09T02:48:29.993 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:29.992+0000 7fa69a249080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-1/keyring: can't open /var/lib/ceph/osd/cluster2-1/keyring: (2) No such file or directory 2021-03-09T02:48:29.994 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:29.992+0000 7fa69a249080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-1/keyring 2021-03-09T02:48:29.994 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:29.992+0000 7fa69a249080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-03-09T02:48:29.994 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:29.992+0000 7fa69a249080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-03-09T02:48:29.994 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:29.992+0000 7fa69a249080 -1 bluestore(/var/lib/ceph/osd/cluster2-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-1/block: (2) No such file or directory 2021-03-09T02:48:29.995 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:29.992+0000 7fa69a249080 -1 bluestore(/var/lib/ceph/osd/cluster2-1) _read_fsid unparsable uuid 2021-03-09T02:48:32.956 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-1 2021-03-09T02:48:33.014 DEBUG:teuthology.orchestra.run.smithi144:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster cluster2 --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/cluster2.monmap 2021-03-09T02:48:33.093 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:33.092+0000 7fcb795b9080 -1 auth: error reading file: /var/lib/ceph/osd/cluster2-2/keyring: can't open /var/lib/ceph/osd/cluster2-2/keyring: (2) No such file or directory 2021-03-09T02:48:33.093 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:33.092+0000 7fcb795b9080 -1 created new key in keyring /var/lib/ceph/osd/cluster2-2/keyring 2021-03-09T02:48:33.094 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:33.092+0000 7fcb795b9080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-03-09T02:48:33.094 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:33.092+0000 7fcb795b9080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-03-09T02:48:33.094 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:33.092+0000 7fcb795b9080 -1 bluestore(/var/lib/ceph/osd/cluster2-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/cluster2-2/block: (2) No such file or directory 2021-03-09T02:48:33.095 INFO:teuthology.orchestra.run.smithi144.stderr:2021-03-09T02:48:33.092+0000 7fcb795b9080 -1 bluestore(/var/lib/ceph/osd/cluster2-2) _read_fsid unparsable uuid 2021-03-09T02:48:36.067 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/osd/cluster2-2 2021-03-09T02:48:36.126 INFO:tasks.ceph:Reading keys from all nodes... 2021-03-09T02:48:36.127 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:36.127 DEBUG:teuthology.orchestra.run.smithi092:> sudo dd if=/var/lib/ceph/mgr/cluster2-x/keyring of=/dev/stdout 2021-03-09T02:48:36.144 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:36.144 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/cluster2-0/keyring of=/dev/stdout 2021-03-09T02:48:36.182 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:36.182 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/cluster2-1/keyring of=/dev/stdout 2021-03-09T02:48:36.235 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:36.236 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/cluster2-2/keyring of=/dev/stdout 2021-03-09T02:48:36.293 DEBUG:teuthology.orchestra.run.smithi092:> set -ex 2021-03-09T02:48:36.294 DEBUG:teuthology.orchestra.run.smithi092:> dd if=/etc/ceph/cluster2.client.0.keyring of=/dev/stdout 2021-03-09T02:48:36.301 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:36.301 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster2.client.mirror.keyring of=/dev/stdout 2021-03-09T02:48:36.341 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:36.342 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster2.client.mirror.0.keyring of=/dev/stdout 2021-03-09T02:48:36.389 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:36.389 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster2.client.mirror.1.keyring of=/dev/stdout 2021-03-09T02:48:36.437 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:36.438 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster2.client.mirror.2.keyring of=/dev/stdout 2021-03-09T02:48:36.485 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:36.485 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster2.client.mirror.3.keyring of=/dev/stdout 2021-03-09T02:48:36.533 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:36.533 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster2.client.mirror.4.keyring of=/dev/stdout 2021-03-09T02:48:36.581 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:36.581 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster2.client.mirror.5.keyring of=/dev/stdout 2021-03-09T02:48:36.629 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:36.629 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/cluster2.client.mirror.6.keyring of=/dev/stdout 2021-03-09T02:48:36.681 INFO:tasks.ceph:Adding keys to all mons... 2021-03-09T02:48:36.681 DEBUG:teuthology.orchestra.run.smithi144:> sudo tee -a /etc/ceph/cluster2.keyring 2021-03-09T02:48:36.736 INFO:teuthology.orchestra.run.smithi144.stdout:[mgr.x] 2021-03-09T02:48:36.737 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD44UZgsh/+BxAAim1Ny1LeT3uSX7UWdIfhJA== 2021-03-09T02:48:36.737 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.0] 2021-03-09T02:48:36.737 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD64UZgCFb1NxAASKJ9Ch87a9Slt+oVhKmRCA== 2021-03-09T02:48:36.737 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.1] 2021-03-09T02:48:36.738 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD94UZgp0FGOxAAeumiww5aHZ/4Ccao9KgsOw== 2021-03-09T02:48:36.738 INFO:teuthology.orchestra.run.smithi144.stdout:[osd.2] 2021-03-09T02:48:36.738 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQAB4kZgsuKdBRAAthOo8YQk2f70YvhFsH80vw== 2021-03-09T02:48:36.738 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2021-03-09T02:48:36.739 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD44UZg9d2ZDBAAppfuc5qBOz5QnUsjHHHJNg== 2021-03-09T02:48:36.739 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror] 2021-03-09T02:48:36.739 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD44UZgDeIfDxAAfe9SdZchO1J8k2/SA1hhhQ== 2021-03-09T02:48:36.739 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.0] 2021-03-09T02:48:36.740 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD44UZg991bFBAAylRLXAFN7xrqbckq0LUpAg== 2021-03-09T02:48:36.740 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.1] 2021-03-09T02:48:36.740 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD44UZg8/alGRAARthBMugVVcHK/eK/6/dlIw== 2021-03-09T02:48:36.740 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.2] 2021-03-09T02:48:36.741 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD44UZgHhfaHhAAWIMOISqzegrULniyteg7Lg== 2021-03-09T02:48:36.741 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.3] 2021-03-09T02:48:36.741 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD44UZgqkoOJBAAZ94z/jP6MLcZnnw45ENRcA== 2021-03-09T02:48:36.741 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.4] 2021-03-09T02:48:36.742 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD44UZgwMtXKRAAi1JAU1gOJCs1tAwhw6ouww== 2021-03-09T02:48:36.742 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.5] 2021-03-09T02:48:36.742 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD44UZgqZ+cLhAA3wmqdkL1N00HO2FtUAKlgA== 2021-03-09T02:48:36.742 INFO:teuthology.orchestra.run.smithi144.stdout:[client.mirror.6] 2021-03-09T02:48:36.743 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQD44UZgUbrKMxAAs4Az5qThvN/YPwZHEr6dMQ== 2021-03-09T02:48:36.744 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2021-03-09T02:48:36.829 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-09T02:48:36.913 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-09T02:48:36.998 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2021-03-09T02:48:37.086 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:37.172 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:37.260 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:37.300 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:37.389 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:37.479 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:37.568 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.4 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:37.610 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.5 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:37.696 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/cluster2.keyring --name=client.mirror.6 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2021-03-09T02:48:37.784 INFO:tasks.ceph:Running mkfs on mon nodes... 2021-03-09T02:48:37.784 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /var/lib/ceph/mon/cluster2-a 2021-03-09T02:48:37.799 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster cluster2 --mkfs -i a --monmap /home/ubuntu/cephtest/cluster2.monmap --keyring /etc/ceph/cluster2.keyring 2021-03-09T02:48:37.917 DEBUG:teuthology.orchestra.run.smithi144:> sudo chown -R ceph:ceph /var/lib/ceph/mon/cluster2-a 2021-03-09T02:48:37.933 DEBUG:teuthology.orchestra.run.smithi144:> rm -- /home/ubuntu/cephtest/cluster2.monmap 2021-03-09T02:48:37.981 INFO:tasks.ceph:Starting mon daemons in cluster cluster2... 2021-03-09T02:48:37.981 INFO:tasks.ceph.cluster2.mon.a:Restarting daemon 2021-03-09T02:48:37.982 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster cluster2 -i a 2021-03-09T02:48:38.025 INFO:tasks.ceph.cluster2.mon.a:Started 2021-03-09T02:48:38.025 INFO:tasks.ceph:Starting mgr daemons in cluster cluster2... 2021-03-09T02:48:38.026 INFO:tasks.ceph.cluster2.mgr.x:Restarting daemon 2021-03-09T02:48:38.026 DEBUG:teuthology.orchestra.run.smithi092:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster cluster2 -i x 2021-03-09T02:48:38.029 INFO:tasks.ceph.cluster2.mgr.x:Started 2021-03-09T02:48:38.029 INFO:tasks.ceph:Setting crush tunables to default 2021-03-09T02:48:38.030 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster cluster2 osd crush tunables default 2021-03-09T02:48:38.343 INFO:teuthology.orchestra.run.smithi144.stderr:adjusted tunables profile to default 2021-03-09T02:48:38.425 INFO:tasks.ceph:Starting osd daemons in cluster cluster2... 2021-03-09T02:48:38.426 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:38.426 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/cluster2-0/fsid of=/dev/stdout 2021-03-09T02:48:38.442 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:38.442 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/cluster2-1/fsid of=/dev/stdout 2021-03-09T02:48:38.500 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:48:38.500 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/osd/cluster2-2/fsid of=/dev/stdout 2021-03-09T02:48:38.557 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster cluster2 osd new dc13e259-5d56-4f77-9fe0-4111f8e96611 0 2021-03-09T02:48:38.829 INFO:teuthology.orchestra.run.smithi144.stdout:0 2021-03-09T02:48:38.848 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster cluster2 osd new 3c00cd12-e949-4db4-b830-da10f9b75e89 1 2021-03-09T02:48:39.067 INFO:teuthology.orchestra.run.smithi144.stdout:1 2021-03-09T02:48:39.081 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster cluster2 osd new 07fb0af1-5636-47b6-98af-14ffc642cfe2 2 2021-03-09T02:48:39.310 INFO:teuthology.orchestra.run.smithi144.stdout:2 2021-03-09T02:48:39.324 INFO:tasks.ceph.cluster2.osd.0:Restarting daemon 2021-03-09T02:48:39.324 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 0 2021-03-09T02:48:39.327 INFO:tasks.ceph.cluster2.osd.0:Started 2021-03-09T02:48:39.328 INFO:tasks.ceph.cluster2.osd.1:Restarting daemon 2021-03-09T02:48:39.328 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 1 2021-03-09T02:48:39.331 INFO:tasks.ceph.cluster2.osd.1:Started 2021-03-09T02:48:39.331 INFO:tasks.ceph.cluster2.osd.2:Restarting daemon 2021-03-09T02:48:39.331 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster cluster2 -i 2 2021-03-09T02:48:39.333 INFO:tasks.ceph.cluster2.osd.2:Started 2021-03-09T02:48:39.334 INFO:tasks.ceph:Waiting for OSDs to come up 2021-03-09T02:48:39.636 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2021-03-09T02:48:39.696 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T02:48:39.696+0000 7fc0f7951080 -1 Falling back to public interface 2021-03-09T02:48:39.835 INFO:teuthology.misc.health.smithi144.stdout: 2021-03-09T02:48:39.836 INFO:teuthology.misc.health.smithi144.stdout:{"epoch":5,"fsid":"f298a705-d51f-4162-ac79-39dab9e11f91","created":"2021-03-09T02:48:38.146273+0000","modified":"2021-03-09T02:48:39.307565+0000","last_up_change":"0.000000","last_in_change":"2021-03-09T02:48:39.307565+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"dc13e259-5d56-4f77-9fe0-4111f8e96611","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"3c00cd12-e949-4db4-b830-da10f9b75e89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"07fb0af1-5636-47b6-98af-14ffc642cfe2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-03-09T02:48:39.848 DEBUG:teuthology.misc:0 of 3 OSDs are up 2021-03-09T02:48:39.949 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T02:48:39.948+0000 7f0840c80080 -1 Falling back to public interface 2021-03-09T02:48:39.962 INFO:tasks.ceph.cluster2.osd.0.smithi144.stderr:2021-03-09T02:48:39.960+0000 7f0e58d30080 -1 Falling back to public interface 2021-03-09T02:48:43.191 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T02:48:43.192+0000 7fc0f7951080 -1 osd.1 0 log_to_monitors {default=true} 2021-03-09T02:48:43.638 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T02:48:43.636+0000 7f0840c80080 -1 osd.2 0 log_to_monitors {default=true} 2021-03-09T02:48:43.745 INFO:tasks.ceph.cluster2.osd.0.smithi144.stderr:2021-03-09T02:48:43.744+0000 7f0e58d30080 -1 osd.0 0 log_to_monitors {default=true} 2021-03-09T02:48:45.168 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T02:48:45.168+0000 7fc0eee1a700 -1 osd.1 0 waiting for initial osdmap 2021-03-09T02:48:45.170 INFO:tasks.ceph.cluster2.osd.0.smithi144.stderr:2021-03-09T02:48:45.168+0000 7f0e501f9700 -1 osd.0 0 waiting for initial osdmap 2021-03-09T02:48:45.170 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T02:48:45.168+0000 7f0838149700 -1 osd.2 0 waiting for initial osdmap 2021-03-09T02:48:46.152 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster cluster2 osd dump --format=json 2021-03-09T02:48:46.445 INFO:teuthology.misc.health.smithi144.stdout: 2021-03-09T02:48:46.446 INFO:teuthology.misc.health.smithi144.stdout:{"epoch":8,"fsid":"f298a705-d51f-4162-ac79-39dab9e11f91","created":"2021-03-09T02:48:38.146273+0000","modified":"2021-03-09T02:48:46.161494+0000","last_up_change":"2021-03-09T02:48:46.161494+0000","last_in_change":"2021-03-09T02:48:39.307565+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"dc13e259-5d56-4f77-9fe0-4111f8e96611","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":16770},{"type":"v1","addr":"172.21.15.144:6817","nonce":16770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":16770},{"type":"v1","addr":"172.21.15.144:6819","nonce":16770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":16770},{"type":"v1","addr":"172.21.15.144:6823","nonce":16770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":16770},{"type":"v1","addr":"172.21.15.144:6821","nonce":16770}]},"public_addr":"172.21.15.144:6817/16770","cluster_addr":"172.21.15.144:6819/16770","heartbeat_back_addr":"172.21.15.144:6823/16770","heartbeat_front_addr":"172.21.15.144:6821/16770","state":["exists","up"]},{"osd":1,"uuid":"3c00cd12-e949-4db4-b830-da10f9b75e89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":16768},{"type":"v1","addr":"172.21.15.144:6801","nonce":16768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":16768},{"type":"v1","addr":"172.21.15.144:6803","nonce":16768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":16768},{"type":"v1","addr":"172.21.15.144:6807","nonce":16768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":16768},{"type":"v1","addr":"172.21.15.144:6805","nonce":16768}]},"public_addr":"172.21.15.144:6801/16768","cluster_addr":"172.21.15.144:6803/16768","heartbeat_back_addr":"172.21.15.144:6807/16768","heartbeat_front_addr":"172.21.15.144:6805/16768","state":["exists","up"]},{"osd":2,"uuid":"07fb0af1-5636-47b6-98af-14ffc642cfe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":16769},{"type":"v1","addr":"172.21.15.144:6809","nonce":16769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":16769},{"type":"v1","addr":"172.21.15.144:6811","nonce":16769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":16769},{"type":"v1","addr":"172.21.15.144:6815","nonce":16769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":16769},{"type":"v1","addr":"172.21.15.144:6813","nonce":16769}]},"public_addr":"172.21.15.144:6809/16769","cluster_addr":"172.21.15.144:6811/16769","heartbeat_back_addr":"172.21.15.144:6815/16769","heartbeat_front_addr":"172.21.15.144:6813/16769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-03-09T02:48:46.459 DEBUG:teuthology.misc:3 of 3 OSDs are up 2021-03-09T02:48:46.459 INFO:tasks.ceph:Creating RBD pool 2021-03-09T02:48:46.460 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster cluster2 osd pool create rbd 8 2021-03-09T02:48:47.184 INFO:teuthology.orchestra.run.smithi144.stderr:pool 'rbd' created 2021-03-09T02:48:47.198 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --cluster cluster2 osd pool application enable rbd rbd --yes-i-really-mean-it 2021-03-09T02:48:48.195 INFO:teuthology.orchestra.run.smithi144.stderr:enabled application 'rbd' on pool 'rbd' 2021-03-09T02:48:48.211 INFO:tasks.ceph:Starting mds daemons in cluster cluster2... 2021-03-09T02:48:48.212 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-03-09T02:48:48.213 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2021-03-09T02:48:48.499 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:48.500 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":10,"fsid":"f298a705-d51f-4162-ac79-39dab9e11f91","created":"2021-03-09T02:48:38.146273+0000","modified":"2021-03-09T02:48:48.178777+0000","last_up_change":"2021-03-09T02:48:46.161494+0000","last_in_change":"2021-03-09T02:48:39.307565+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-09T02:48:46.234030+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-09T02:48:46.824730+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc13e259-5d56-4f77-9fe0-4111f8e96611","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":16770},{"type":"v1","addr":"172.21.15.144:6817","nonce":16770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":16770},{"type":"v1","addr":"172.21.15.144:6819","nonce":16770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":16770},{"type":"v1","addr":"172.21.15.144:6823","nonce":16770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":16770},{"type":"v1","addr":"172.21.15.144:6821","nonce":16770}]},"public_addr":"172.21.15.144:6817/16770","cluster_addr":"172.21.15.144:6819/16770","heartbeat_back_addr":"172.21.15.144:6823/16770","heartbeat_front_addr":"172.21.15.144:6821/16770","state":["exists","up"]},{"osd":1,"uuid":"3c00cd12-e949-4db4-b830-da10f9b75e89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":16768},{"type":"v1","addr":"172.21.15.144:6801","nonce":16768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":16768},{"type":"v1","addr":"172.21.15.144:6803","nonce":16768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":16768},{"type":"v1","addr":"172.21.15.144:6807","nonce":16768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":16768},{"type":"v1","addr":"172.21.15.144:6805","nonce":16768}]},"public_addr":"172.21.15.144:6801/16768","cluster_addr":"172.21.15.144:6803/16768","heartbeat_back_addr":"172.21.15.144:6807/16768","heartbeat_front_addr":"172.21.15.144:6805/16768","state":["exists","up"]},{"osd":2,"uuid":"07fb0af1-5636-47b6-98af-14ffc642cfe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":16769},{"type":"v1","addr":"172.21.15.144:6809","nonce":16769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":16769},{"type":"v1","addr":"172.21.15.144:6811","nonce":16769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":16769},{"type":"v1","addr":"172.21.15.144:6815","nonce":16769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":16769},{"type":"v1","addr":"172.21.15.144:6813","nonce":16769}]},"public_addr":"172.21.15.144:6809/16769","cluster_addr":"172.21.15.144:6811/16769","heartbeat_back_addr":"172.21.15.144:6815/16769","heartbeat_front_addr":"172.21.15.144:6813/16769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:44.795044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:44.199117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:44.688003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-03-09T02:48:48.514 INFO:tasks.ceph.ceph_manager.cluster2:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-03-09T02:48:46.234030+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2021-03-09T02:48:46.824730+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '10', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2021-03-09T02:48:48.514 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd pool get device_health_metrics pg_num 2021-03-09T02:48:48.817 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2021-03-09T02:48:48.834 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd pool get rbd pg_num 2021-03-09T02:48:49.135 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 8 2021-03-09T02:48:49.151 INFO:tasks.ceph:Waiting until cluster2 daemons up and pgs clean... 2021-03-09T02:48:49.151 INFO:tasks.ceph.ceph_manager.cluster2:waiting for mgr available 2021-03-09T02:48:49.152 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 mgr dump --format=json 2021-03-09T02:48:49.493 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:49.506 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.92:6826","nonce":19928},{"type":"v1","addr":"172.21.15.92:6827","nonce":19928}]},"active_addr":"172.21.15.92:6827/19928","active_change":"2021-03-09T02:48:42.085855+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":3994829229}]},{"addrvec":[{"type":"v2","addr":"172.21.15.92:0","nonce":1813839751}]}]}} 2021-03-09T02:48:49.508 INFO:tasks.ceph.ceph_manager.cluster2:mgr available! 2021-03-09T02:48:49.509 INFO:tasks.ceph.ceph_manager.cluster2:waiting for all up 2021-03-09T02:48:49.509 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-03-09T02:48:49.814 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:49.814 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":11,"fsid":"f298a705-d51f-4162-ac79-39dab9e11f91","created":"2021-03-09T02:48:38.146273+0000","modified":"2021-03-09T02:48:49.192051+0000","last_up_change":"2021-03-09T02:48:46.161494+0000","last_in_change":"2021-03-09T02:48:39.307565+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-09T02:48:46.234030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-09T02:48:46.824730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc13e259-5d56-4f77-9fe0-4111f8e96611","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":16770},{"type":"v1","addr":"172.21.15.144:6817","nonce":16770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":16770},{"type":"v1","addr":"172.21.15.144:6819","nonce":16770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":16770},{"type":"v1","addr":"172.21.15.144:6823","nonce":16770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":16770},{"type":"v1","addr":"172.21.15.144:6821","nonce":16770}]},"public_addr":"172.21.15.144:6817/16770","cluster_addr":"172.21.15.144:6819/16770","heartbeat_back_addr":"172.21.15.144:6823/16770","heartbeat_front_addr":"172.21.15.144:6821/16770","state":["exists","up"]},{"osd":1,"uuid":"3c00cd12-e949-4db4-b830-da10f9b75e89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":16768},{"type":"v1","addr":"172.21.15.144:6801","nonce":16768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":16768},{"type":"v1","addr":"172.21.15.144:6803","nonce":16768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":16768},{"type":"v1","addr":"172.21.15.144:6807","nonce":16768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":16768},{"type":"v1","addr":"172.21.15.144:6805","nonce":16768}]},"public_addr":"172.21.15.144:6801/16768","cluster_addr":"172.21.15.144:6803/16768","heartbeat_back_addr":"172.21.15.144:6807/16768","heartbeat_front_addr":"172.21.15.144:6805/16768","state":["exists","up"]},{"osd":2,"uuid":"07fb0af1-5636-47b6-98af-14ffc642cfe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":16769},{"type":"v1","addr":"172.21.15.144:6809","nonce":16769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":16769},{"type":"v1","addr":"172.21.15.144:6811","nonce":16769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":16769},{"type":"v1","addr":"172.21.15.144:6815","nonce":16769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":16769},{"type":"v1","addr":"172.21.15.144:6813","nonce":16769}]},"public_addr":"172.21.15.144:6809/16769","cluster_addr":"172.21.15.144:6811/16769","heartbeat_back_addr":"172.21.15.144:6815/16769","heartbeat_front_addr":"172.21.15.144:6813/16769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:44.795044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:44.199117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:44.688003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-03-09T02:48:49.829 INFO:tasks.ceph.ceph_manager.cluster2:all up! 2021-03-09T02:48:49.829 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd dump --format=json 2021-03-09T02:48:50.126 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:50.126 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":11,"fsid":"f298a705-d51f-4162-ac79-39dab9e11f91","created":"2021-03-09T02:48:38.146273+0000","modified":"2021-03-09T02:48:49.192051+0000","last_up_change":"2021-03-09T02:48:46.161494+0000","last_in_change":"2021-03-09T02:48:39.307565+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-03-09T02:48:46.234030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2021-03-09T02:48:46.824730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"dc13e259-5d56-4f77-9fe0-4111f8e96611","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":16770},{"type":"v1","addr":"172.21.15.144:6817","nonce":16770}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":16770},{"type":"v1","addr":"172.21.15.144:6819","nonce":16770}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":16770},{"type":"v1","addr":"172.21.15.144:6823","nonce":16770}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":16770},{"type":"v1","addr":"172.21.15.144:6821","nonce":16770}]},"public_addr":"172.21.15.144:6817/16770","cluster_addr":"172.21.15.144:6819/16770","heartbeat_back_addr":"172.21.15.144:6823/16770","heartbeat_front_addr":"172.21.15.144:6821/16770","state":["exists","up"]},{"osd":1,"uuid":"3c00cd12-e949-4db4-b830-da10f9b75e89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":16768},{"type":"v1","addr":"172.21.15.144:6801","nonce":16768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":16768},{"type":"v1","addr":"172.21.15.144:6803","nonce":16768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":16768},{"type":"v1","addr":"172.21.15.144:6807","nonce":16768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":16768},{"type":"v1","addr":"172.21.15.144:6805","nonce":16768}]},"public_addr":"172.21.15.144:6801/16768","cluster_addr":"172.21.15.144:6803/16768","heartbeat_back_addr":"172.21.15.144:6807/16768","heartbeat_front_addr":"172.21.15.144:6805/16768","state":["exists","up"]},{"osd":2,"uuid":"07fb0af1-5636-47b6-98af-14ffc642cfe2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":16769},{"type":"v1","addr":"172.21.15.144:6809","nonce":16769}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":16769},{"type":"v1","addr":"172.21.15.144:6811","nonce":16769}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":16769},{"type":"v1","addr":"172.21.15.144:6815","nonce":16769}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":16769},{"type":"v1","addr":"172.21.15.144:6813","nonce":16769}]},"public_addr":"172.21.15.144:6809/16769","cluster_addr":"172.21.15.144:6811/16769","heartbeat_back_addr":"172.21.15.144:6815/16769","heartbeat_front_addr":"172.21.15.144:6813/16769","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:44.795044+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:44.199117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-03-09T02:48:44.688003+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-03-09T02:48:50.140 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.0 flush_pg_stats 2021-03-09T02:48:50.311 INFO:teuthology.orchestra.run.smithi144.stdout:34359738371 2021-03-09T02:48:50.312 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.1 flush_pg_stats 2021-03-09T02:48:50.483 INFO:teuthology.orchestra.run.smithi144.stdout:34359738370 2021-03-09T02:48:50.484 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 tell osd.2 flush_pg_stats 2021-03-09T02:48:50.651 INFO:teuthology.orchestra.run.smithi144.stdout:34359738371 2021-03-09T02:48:50.651 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2021-03-09T02:48:50.951 INFO:teuthology.orchestra.run.smithi144.stdout:34359738370 2021-03-09T02:48:50.966 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738370 for osd.0 2021-03-09T02:48:51.968 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.0 2021-03-09T02:48:52.281 INFO:teuthology.orchestra.run.smithi144.stdout:34359738372 2021-03-09T02:48:52.296 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738372 for osd.0 2021-03-09T02:48:52.297 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.1 2021-03-09T02:48:52.599 INFO:teuthology.orchestra.run.smithi144.stdout:34359738371 2021-03-09T02:48:52.614 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738370 got 34359738371 for osd.1 2021-03-09T02:48:52.614 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 osd last-stat-seq osd.2 2021-03-09T02:48:52.911 INFO:teuthology.orchestra.run.smithi144.stdout:34359738372 2021-03-09T02:48:52.926 INFO:tasks.ceph.ceph_manager.cluster2:need seq 34359738371 got 34359738372 for osd.2 2021-03-09T02:48:52.926 INFO:tasks.ceph.ceph_manager.cluster2:waiting for clean 2021-03-09T02:48:52.926 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2021-03-09T02:48:53.225 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:53.227 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2021-03-09T02:48:53.239 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-03-09T02:48:52.099137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14628,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100892,"statfs":{"total":289910292480,"available":289895313408,"internally_reserved":0,"allocated":274432,"data_stored":71260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.199758+0000","last_change":"2021-03-09T02:48:48.199758+0000","last_active":"2021-03-09T02:48:48.199758+0000","last_peered":"2021-03-09T02:48:48.199758+0000","last_clean":"2021-03-09T02:48:48.199758+0000","last_became_active":"2021-03-09T02:48:48.199395+0000","last_became_peered":"2021-03-09T02:48:48.199395+0000","last_unstale":"2021-03-09T02:48:48.199758+0000","last_undegraded":"2021-03-09T02:48:48.199758+0000","last_fullsized":"2021-03-09T02:48:48.199758+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.201812+0000","last_change":"2021-03-09T02:48:48.201812+0000","last_active":"2021-03-09T02:48:48.201812+0000","last_peered":"2021-03-09T02:48:48.201812+0000","last_clean":"2021-03-09T02:48:48.201812+0000","last_became_active":"2021-03-09T02:48:48.201431+0000","last_became_peered":"2021-03-09T02:48:48.201431+0000","last_unstale":"2021-03-09T02:48:48.201812+0000","last_undegraded":"2021-03-09T02:48:48.201812+0000","last_fullsized":"2021-03-09T02:48:48.201812+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.201794+0000","last_change":"2021-03-09T02:48:48.201794+0000","last_active":"2021-03-09T02:48:48.201794+0000","last_peered":"2021-03-09T02:48:48.201794+0000","last_clean":"2021-03-09T02:48:48.201794+0000","last_became_active":"2021-03-09T02:48:48.201420+0000","last_became_peered":"2021-03-09T02:48:48.201420+0000","last_unstale":"2021-03-09T02:48:48.201794+0000","last_undegraded":"2021-03-09T02:48:48.201794+0000","last_fullsized":"2021-03-09T02:48:48.201794+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.200726+0000","last_change":"2021-03-09T02:48:48.200726+0000","last_active":"2021-03-09T02:48:48.200726+0000","last_peered":"2021-03-09T02:48:48.200726+0000","last_clean":"2021-03-09T02:48:48.200726+0000","last_became_active":"2021-03-09T02:48:48.200196+0000","last_became_peered":"2021-03-09T02:48:48.200196+0000","last_unstale":"2021-03-09T02:48:48.200726+0000","last_undegraded":"2021-03-09T02:48:48.200726+0000","last_fullsized":"2021-03-09T02:48:48.200726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.198574+0000","last_change":"2021-03-09T02:48:48.198574+0000","last_active":"2021-03-09T02:48:48.198574+0000","last_peered":"2021-03-09T02:48:48.198574+0000","last_clean":"2021-03-09T02:48:48.198574+0000","last_became_active":"2021-03-09T02:48:48.198312+0000","last_became_peered":"2021-03-09T02:48:48.198312+0000","last_unstale":"2021-03-09T02:48:48.198574+0000","last_undegraded":"2021-03-09T02:48:48.198574+0000","last_fullsized":"2021-03-09T02:48:48.198574+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.202223+0000","last_change":"2021-03-09T02:48:48.202223+0000","last_active":"2021-03-09T02:48:48.202223+0000","last_peered":"2021-03-09T02:48:48.202223+0000","last_clean":"2021-03-09T02:48:48.202223+0000","last_became_active":"2021-03-09T02:48:48.201898+0000","last_became_peered":"2021-03-09T02:48:48.201898+0000","last_unstale":"2021-03-09T02:48:48.202223+0000","last_undegraded":"2021-03-09T02:48:48.202223+0000","last_fullsized":"2021-03-09T02:48:48.202223+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.203119+0000","last_change":"2021-03-09T02:48:48.203119+0000","last_active":"2021-03-09T02:48:48.203119+0000","last_peered":"2021-03-09T02:48:48.203119+0000","last_clean":"2021-03-09T02:48:48.203119+0000","last_became_active":"2021-03-09T02:48:48.202797+0000","last_became_peered":"2021-03-09T02:48:48.202797+0000","last_unstale":"2021-03-09T02:48:48.203119+0000","last_undegraded":"2021-03-09T02:48:48.203119+0000","last_fullsized":"2021-03-09T02:48:48.203119+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.204434+0000","last_change":"2021-03-09T02:48:48.203804+0000","last_active":"2021-03-09T02:48:48.204434+0000","last_peered":"2021-03-09T02:48:48.204434+0000","last_clean":"2021-03-09T02:48:48.204434+0000","last_became_active":"2021-03-09T02:48:48.203275+0000","last_became_peered":"2021-03-09T02:48:48.203275+0000","last_unstale":"2021-03-09T02:48:48.204434+0000","last_undegraded":"2021-03-09T02:48:48.204434+0000","last_fullsized":"2021-03-09T02:48:48.204434+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.202138+0000","last_change":"2021-03-09T02:48:48.202138+0000","last_active":"2021-03-09T02:48:48.202138+0000","last_peered":"2021-03-09T02:48:48.202138+0000","last_clean":"2021-03-09T02:48:48.202138+0000","last_became_active":"2021-03-09T02:48:48.201715+0000","last_became_peered":"2021-03-09T02:48:48.201715+0000","last_unstale":"2021-03-09T02:48:48.202138+0000","last_undegraded":"2021-03-09T02:48:48.202138+0000","last_fullsized":"2021-03-09T02:48:48.202138+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4832,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367008,"statfs":{"total":96636764160,"available":96631816192,"internally_reserved":0,"allocated":90112,"data_stored":22828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4900,"kb_used_data":92,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366940,"statfs":{"total":96636764160,"available":96631746560,"internally_reserved":0,"allocated":94208,"data_stored":25604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4896,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366944,"statfs":{"total":96636764160,"available":96631750656,"internally_reserved":0,"allocated":90112,"data_stored":22828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2021-03-09T02:48:53.240 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 pg dump --format=json 2021-03-09T02:48:53.531 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:53.533 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2021-03-09T02:48:53.544 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2021-03-09T02:48:52.099137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":14628,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":14336,"kb_avail":283100892,"statfs":{"total":289910292480,"available":289895313408,"internally_reserved":0,"allocated":274432,"data_stored":71260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":14680064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.199758+0000","last_change":"2021-03-09T02:48:48.199758+0000","last_active":"2021-03-09T02:48:48.199758+0000","last_peered":"2021-03-09T02:48:48.199758+0000","last_clean":"2021-03-09T02:48:48.199758+0000","last_became_active":"2021-03-09T02:48:48.199395+0000","last_became_peered":"2021-03-09T02:48:48.199395+0000","last_unstale":"2021-03-09T02:48:48.199758+0000","last_undegraded":"2021-03-09T02:48:48.199758+0000","last_fullsized":"2021-03-09T02:48:48.199758+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.201812+0000","last_change":"2021-03-09T02:48:48.201812+0000","last_active":"2021-03-09T02:48:48.201812+0000","last_peered":"2021-03-09T02:48:48.201812+0000","last_clean":"2021-03-09T02:48:48.201812+0000","last_became_active":"2021-03-09T02:48:48.201431+0000","last_became_peered":"2021-03-09T02:48:48.201431+0000","last_unstale":"2021-03-09T02:48:48.201812+0000","last_undegraded":"2021-03-09T02:48:48.201812+0000","last_fullsized":"2021-03-09T02:48:48.201812+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.201794+0000","last_change":"2021-03-09T02:48:48.201794+0000","last_active":"2021-03-09T02:48:48.201794+0000","last_peered":"2021-03-09T02:48:48.201794+0000","last_clean":"2021-03-09T02:48:48.201794+0000","last_became_active":"2021-03-09T02:48:48.201420+0000","last_became_peered":"2021-03-09T02:48:48.201420+0000","last_unstale":"2021-03-09T02:48:48.201794+0000","last_undegraded":"2021-03-09T02:48:48.201794+0000","last_fullsized":"2021-03-09T02:48:48.201794+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.200726+0000","last_change":"2021-03-09T02:48:48.200726+0000","last_active":"2021-03-09T02:48:48.200726+0000","last_peered":"2021-03-09T02:48:48.200726+0000","last_clean":"2021-03-09T02:48:48.200726+0000","last_became_active":"2021-03-09T02:48:48.200196+0000","last_became_peered":"2021-03-09T02:48:48.200196+0000","last_unstale":"2021-03-09T02:48:48.200726+0000","last_undegraded":"2021-03-09T02:48:48.200726+0000","last_fullsized":"2021-03-09T02:48:48.200726+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.198574+0000","last_change":"2021-03-09T02:48:48.198574+0000","last_active":"2021-03-09T02:48:48.198574+0000","last_peered":"2021-03-09T02:48:48.198574+0000","last_clean":"2021-03-09T02:48:48.198574+0000","last_became_active":"2021-03-09T02:48:48.198312+0000","last_became_peered":"2021-03-09T02:48:48.198312+0000","last_unstale":"2021-03-09T02:48:48.198574+0000","last_undegraded":"2021-03-09T02:48:48.198574+0000","last_fullsized":"2021-03-09T02:48:48.198574+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.202223+0000","last_change":"2021-03-09T02:48:48.202223+0000","last_active":"2021-03-09T02:48:48.202223+0000","last_peered":"2021-03-09T02:48:48.202223+0000","last_clean":"2021-03-09T02:48:48.202223+0000","last_became_active":"2021-03-09T02:48:48.201898+0000","last_became_peered":"2021-03-09T02:48:48.201898+0000","last_unstale":"2021-03-09T02:48:48.202223+0000","last_undegraded":"2021-03-09T02:48:48.202223+0000","last_fullsized":"2021-03-09T02:48:48.202223+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.203119+0000","last_change":"2021-03-09T02:48:48.203119+0000","last_active":"2021-03-09T02:48:48.203119+0000","last_peered":"2021-03-09T02:48:48.203119+0000","last_clean":"2021-03-09T02:48:48.203119+0000","last_became_active":"2021-03-09T02:48:48.202797+0000","last_became_peered":"2021-03-09T02:48:48.202797+0000","last_unstale":"2021-03-09T02:48:48.203119+0000","last_undegraded":"2021-03-09T02:48:48.203119+0000","last_fullsized":"2021-03-09T02:48:48.203119+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"9","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.204434+0000","last_change":"2021-03-09T02:48:48.203804+0000","last_active":"2021-03-09T02:48:48.204434+0000","last_peered":"2021-03-09T02:48:48.204434+0000","last_clean":"2021-03-09T02:48:48.204434+0000","last_became_active":"2021-03-09T02:48:48.203275+0000","last_became_peered":"2021-03-09T02:48:48.203275+0000","last_unstale":"2021-03-09T02:48:48.204434+0000","last_undegraded":"2021-03-09T02:48:48.204434+0000","last_fullsized":"2021-03-09T02:48:48.204434+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"8","reported_epoch":"10","state":"active+clean","last_fresh":"2021-03-09T02:48:48.202138+0000","last_change":"2021-03-09T02:48:48.202138+0000","last_active":"2021-03-09T02:48:48.202138+0000","last_peered":"2021-03-09T02:48:48.202138+0000","last_clean":"2021-03-09T02:48:48.202138+0000","last_became_active":"2021-03-09T02:48:48.201715+0000","last_became_peered":"2021-03-09T02:48:48.201715+0000","last_unstale":"2021-03-09T02:48:48.202138+0000","last_undegraded":"2021-03-09T02:48:48.202138+0000","last_fullsized":"2021-03-09T02:48:48.202138+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-03-09T02:48:47.170069+0000","last_clean_scrub_stamp":"2021-03-09T02:48:47.170069+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4832,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":94367008,"statfs":{"total":96636764160,"available":96631816192,"internally_reserved":0,"allocated":90112,"data_stored":22828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4900,"kb_used_data":92,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366940,"statfs":{"total":96636764160,"available":96631746560,"internally_reserved":0,"allocated":94208,"data_stored":25604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4896,"kb_used_data":88,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366944,"statfs":{"total":96636764160,"available":96631750656,"internally_reserved":0,"allocated":90112,"data_stored":22828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2021-03-09T02:48:53.546 INFO:tasks.ceph.ceph_manager.cluster2:clean! 2021-03-09T02:48:53.546 INFO:tasks.ceph:Waiting until ceph cluster cluster2 is healthy... 2021-03-09T02:48:53.546 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy 2021-03-09T02:48:53.547 DEBUG:teuthology.orchestra.run.smithi144:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster cluster2 health --format=json 2021-03-09T02:48:53.904 INFO:teuthology.orchestra.run.smithi144.stdout: 2021-03-09T02:48:53.918 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-03-09T02:48:53.918 INFO:tasks.ceph.ceph_manager.cluster2:wait_until_healthy done 2021-03-09T02:48:53.919 INFO:teuthology.run_tasks:Running task exec... 2021-03-09T02:48:53.932 INFO:teuthology.task.exec:Executing custom commands... 2021-03-09T02:48:53.932 INFO:teuthology.task.exec:Running commands on role cluster1.client.mirror.0 host ubuntu@smithi144.front.sepia.ceph.com 2021-03-09T02:48:53.933 DEBUG:teuthology.orchestra.run.smithi144:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2021-03-09T02:48:54.320 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.mirror 2021-03-09T02:48:54.334 DEBUG:teuthology.orchestra.run.smithi144:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-03-09T02:48:54.716 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.mirror.0 2021-03-09T02:48:54.729 DEBUG:teuthology.orchestra.run.smithi144:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-03-09T02:48:55.103 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.mirror.1 2021-03-09T02:48:55.116 DEBUG:teuthology.orchestra.run.smithi144:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-03-09T02:48:55.489 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.mirror.2 2021-03-09T02:48:55.503 DEBUG:teuthology.orchestra.run.smithi144:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster1 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-03-09T02:48:55.881 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.mirror.3 2021-03-09T02:48:55.894 DEBUG:teuthology.orchestra.run.smithi144:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror mon '"'"'profile rbd-mirror-peer'"'"' osd '"'"'profile rbd'"'"'' 2021-03-09T02:48:56.269 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.mirror 2021-03-09T02:48:56.280 DEBUG:teuthology.orchestra.run.smithi144:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.0 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-03-09T02:48:56.634 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.mirror.0 2021-03-09T02:48:56.646 DEBUG:teuthology.orchestra.run.smithi144:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.1 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-03-09T02:48:57.016 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.mirror.1 2021-03-09T02:48:57.032 DEBUG:teuthology.orchestra.run.smithi144:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.2 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-03-09T02:48:57.398 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.mirror.2 2021-03-09T02:48:57.410 DEBUG:teuthology.orchestra.run.smithi144:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph --cluster cluster2 auth caps client.mirror.3 mon '"'"'profile rbd-mirror'"'"' osd '"'"'profile rbd'"'"'' 2021-03-09T02:48:57.774 INFO:teuthology.orchestra.run.smithi144.stderr:updated caps for client.mirror.3 2021-03-09T02:48:57.789 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-03-09T02:48:57.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T02:48:57.802 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T02:48:57.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T02:48:57.806 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-03-09T02:48:57.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T02:48:57.825 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T02:48:57.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T02:48:57.832 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-03-09T02:48:57.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T02:48:57.843 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T02:48:57.850 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T02:48:57.851 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-03-09T02:48:57.861 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T02:48:57.861 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T02:48:57.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T02:48:57.873 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-03-09T02:48:57.884 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T02:48:57.885 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T02:48:57.888 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T02:48:57.889 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-03-09T02:48:57.901 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T02:48:57.902 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T02:48:57.904 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T02:48:57.904 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-03-09T02:48:57.918 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T02:48:57.918 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T02:48:57.920 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T02:48:57.921 INFO:teuthology.run_tasks:Running task rbd-mirror... 2021-03-09T02:48:57.936 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T02:48:57.936 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T02:48:57.939 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T02:48:57.939 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2021-03-09T02:48:57.956 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1615258137 2021-03-09T02:48:57.957 DEBUG:tasks.rbd_mirror_thrash:Yielding 2021-03-09T02:48:57.957 INFO:teuthology.run_tasks:Running task rbd-mirror-thrash... 2021-03-09T02:48:57.973 INFO:tasks.rbd_mirror_thrash:rbd_mirror_thrash using random seed: 1615258137 2021-03-09T02:48:57.974 DEBUG:tasks.rbd_mirror_thrash:Yielding 2021-03-09T02:48:57.974 INFO:teuthology.run_tasks:Running task workunit... 2021-03-09T02:48:57.991 INFO:tasks.workunit:Pulling workunits from ref 9e448db9e385343da144613d840349182443aeee 2021-03-09T02:48:57.992 INFO:tasks.workunit:Making a separate scratch dir for every client... 2021-03-09T02:48:57.993 DEBUG:teuthology.orchestra.run.smithi144:> stat -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-03-09T02:48:57.993 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster1 2021-03-09T02:48:57.994 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-03-09T02:48:57.994 INFO:tasks.rbd_mirror_thrash:starting thrash for cluster cluster2 2021-03-09T02:48:57.994 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-03-09T02:48:58.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:48:58.006 INFO:teuthology.orchestra.run.smithi144.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.cluster1.mirror': No such file or directory 2021-03-09T02:48:58.007 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -- /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-03-09T02:48:58.057 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.cluster1.mirror 2021-03-09T02:48:58.057 DEBUG:teuthology.orchestra.run.smithi144:> cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror && mkdir -- client.mirror 2021-03-09T02:48:58.105 INFO:tasks.workunit:timeout=6h 2021-03-09T02:48:58.105 INFO:tasks.workunit:cleanup=True 2021-03-09T02:48:58.106 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/clone.cluster1.client.mirror && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.cluster1.client.mirror && cd /home/ubuntu/cephtest/clone.cluster1.client.mirror && git checkout 9e448db9e385343da144613d840349182443aeee 2021-03-09T02:48:58.154 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Cloning into '/home/ubuntu/cephtest/clone.cluster1.client.mirror'... 2021-03-09T02:49:31.882 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Note: checking out '9e448db9e385343da144613d840349182443aeee'. 2021-03-09T02:49:31.882 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:49:31.882 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2021-03-09T02:49:31.883 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:changes and commit them, and you can discard any commits you make in this 2021-03-09T02:49:31.883 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:state without impacting any branches by performing another checkout. 2021-03-09T02:49:31.883 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:49:31.883 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:If you want to create a new branch to retain commits you create, you may 2021-03-09T02:49:31.883 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:do so (now or later) by using -b with the checkout command again. Example: 2021-03-09T02:49:31.884 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:49:31.884 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: git checkout -b 2021-03-09T02:49:31.884 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:49:31.884 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:HEAD is now at 9e448db9e3 Merge branch 'wip-revert-autoscaler-pacific' of https://github.com/neha-ojha/ceph into wip-yuri5-testing-2021-03-08-1049-pacific 2021-03-09T02:49:31.886 DEBUG:teuthology.orchestra.run.smithi144:> cd -- /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror 2021-03-09T02:49:31.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2021-03-09T02:49:31.938 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2021-03-09T02:49:31.939 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2021-03-09T02:49:32.665 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2021-03-09T02:49:32.797 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2021-03-09T02:49:32.860 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/direct_io' 2021-03-09T02:49:32.862 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2021-03-09T02:49:32.863 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2021-03-09T02:49:32.915 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/fs' 2021-03-09T02:49:32.920 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2021-03-09T02:49:32.920 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/home/ubuntu/cephtest/workunits.list.cluster1.client.mirror of=/dev/stdout 2021-03-09T02:49:32.971 INFO:tasks.workunit:Running workunits matching rbd/rbd_mirror_stress.sh on cluster1.client.mirror... 2021-03-09T02:49:32.972 INFO:tasks.workunit:Running workunit rbd/rbd_mirror_stress.sh... 2021-03-09T02:49:32.973 DEBUG:teuthology.orchestra.run.smithi144:workunit test rbd/rbd_mirror_stress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && cd -- /home/ubuntu/cephtest/mnt.cluster1.mirror/client.mirror/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9e448db9e385343da144613d840349182443aeee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster cluster1" CEPH_ID="mirror" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror CEPH_ARGS='' MIRROR_IMAGE_MODE=snapshot MIRROR_POOL_MODE=image RBD_IMAGE_FEATURES=layering,exclusive-lock RBD_MIRROR_INSTANCES=4 RBD_MIRROR_USE_EXISTING_CLUSTER=1 RBD_MIRROR_USE_RBD_MIRROR=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 6h /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-03-09T02:49:33.024 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ IMAGE_COUNT=50 2021-03-09T02:49:33.025 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ export LOCKDEP=0 2021-03-09T02:49:33.026 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-03-09T02:49:33.026 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ . /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_helpers.sh 2021-03-09T02:49:33.027 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ type xmlstarlet 2021-03-09T02:49:33.027 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ XMLSTARLET=xmlstarlet 2021-03-09T02:49:33.027 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ RBD_MIRROR_INSTANCES=4 2021-03-09T02:49:33.027 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CLUSTER1=cluster1 2021-03-09T02:49:33.028 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CLUSTER2=cluster2 2021-03-09T02:49:33.028 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ PEER_CLUSTER_SUFFIX= 2021-03-09T02:49:33.028 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ POOL=mirror 2021-03-09T02:49:33.028 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ PARENT_POOL=mirror_parent 2021-03-09T02:49:33.028 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ NS1=ns1 2021-03-09T02:49:33.029 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ NS2=ns2 2021-03-09T02:49:33.029 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ TEMPDIR= 2021-03-09T02:49:33.029 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ID=mirror 2021-03-09T02:49:33.029 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ RBD_IMAGE_FEATURES=layering,exclusive-lock 2021-03-09T02:49:33.029 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ MIRROR_USER_ID_PREFIX=mirror. 2021-03-09T02:49:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ MIRROR_POOL_MODE=image 2021-03-09T02:49:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ MIRROR_IMAGE_MODE=snapshot 2021-03-09T02:49:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ export CEPH_ARGS=--id mirror 2021-03-09T02:49:33.030 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ LAST_MIRROR_INSTANCE=3 2021-03-09T02:49:33.031 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ dirname /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/rbd_mirror_stress.sh 2021-03-09T02:49:33.032 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ readlink -f /home/ubuntu/cephtest/clone.cluster1.client.mirror/qa/workunits/rbd/../../../src 2021-03-09T02:49:33.032 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ROOT=/home/ubuntu/cephtest/clone.cluster1.client.mirror/src 2021-03-09T02:49:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_BIN=. 2021-03-09T02:49:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_SRC=. 2021-03-09T02:49:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -e CMakeCache.txt ] 2021-03-09T02:49:33.033 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 -gt 0 ] 2021-03-09T02:49:33.034 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ setup 2021-03-09T02:49:33.034 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local c 2021-03-09T02:49:33.034 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ trap cleanup $? INT TERM EXIT 2021-03-09T02:49:33.034 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ setup_tempdir 2021-03-09T02:49:33.034 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:33.035 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mktemp -d 2021-03-09T02:49:33.035 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ TEMPDIR=/tmp/tmp.y6RrY8NxKl 2021-03-09T02:49:33.035 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -z 1 ] 2021-03-09T02:49:33.035 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ setup_pools cluster1 cluster2 2021-03-09T02:49:33.035 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local remote_cluster=cluster2 2021-03-09T02:49:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mon_map_file 2021-03-09T02:49:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mon_addr 2021-03-09T02:49:33.036 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local admin_key_file 2021-03-09T02:49:33.037 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local uuid 2021-03-09T02:49:33.037 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= ceph --cluster cluster1 osd pool create mirror 64 64 2021-03-09T02:49:34.191 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:pool 'mirror' already exists 2021-03-09T02:49:34.201 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= ceph --cluster cluster1 osd pool create mirror_parent 64 64 2021-03-09T02:49:35.201 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:pool 'mirror_parent' already exists 2021-03-09T02:49:35.210 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster1 pool init mirror 2021-03-09T02:49:38.130 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster1 pool init mirror_parent 2021-03-09T02:49:41.163 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:41.164 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror pool enable --site-name cluster1 mirror image 2021-03-09T02:49:41.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool enable mirror_parent image 2021-03-09T02:49:41.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 namespace create mirror/ns1 2021-03-09T02:49:41.311 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 namespace create mirror/ns2 2021-03-09T02:49:41.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns1 image 2021-03-09T02:49:41.413 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool enable mirror/ns2 image 2021-03-09T02:49:41.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -z ] 2021-03-09T02:49:41.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -z ] 2021-03-09T02:49:41.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ peer_add cluster1 mirror cluster2 2021-03-09T02:49:41.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:41.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:41.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:49:41.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:41.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local client_cluster=cluster2 2021-03-09T02:49:41.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:41.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local remote_cluster=cluster2 2021-03-09T02:49:41.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local uuid_var_name 2021-03-09T02:49:41.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:41.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local error_code 2021-03-09T02:49:41.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local peer_uuid 2021-03-09T02:49:41.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:49:41.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool peer add mirror cluster2 2021-03-09T02:49:41.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ peer_uuid=f1f6649d-caa3-4626-a3e3-ed0c4af5b8b0 2021-03-09T02:49:41.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=0 2021-03-09T02:49:41.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:49:41.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 -eq 17 ] 2021-03-09T02:49:41.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 0 -eq 0 2021-03-09T02:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ peer_add cluster1 mirror_parent cluster2 2021-03-09T02:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror_parent 2021-03-09T02:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local client_cluster=cluster2 2021-03-09T02:49:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:41.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local remote_cluster=cluster2 2021-03-09T02:49:41.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local uuid_var_name 2021-03-09T02:49:41.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:41.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local error_code 2021-03-09T02:49:41.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local peer_uuid 2021-03-09T02:49:41.519 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:49:41.519 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool peer add mirror_parent cluster2 2021-03-09T02:49:41.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ peer_uuid=ed4549e0-2bc1-4f43-80de-3341de0fec6c 2021-03-09T02:49:41.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=0 2021-03-09T02:49:41.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:49:41.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 -eq 17 ] 2021-03-09T02:49:41.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 0 -eq 0 2021-03-09T02:49:41.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:41.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:49:41.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ setup_pools cluster2 cluster1 2021-03-09T02:49:41.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:49:41.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local remote_cluster=cluster1 2021-03-09T02:49:41.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mon_map_file 2021-03-09T02:49:41.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mon_addr 2021-03-09T02:49:41.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local admin_key_file 2021-03-09T02:49:41.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local uuid 2021-03-09T02:49:41.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= ceph --cluster cluster2 osd pool create mirror 64 64 2021-03-09T02:49:42.409 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:pool 'mirror' already exists 2021-03-09T02:49:42.419 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= ceph --cluster cluster2 osd pool create mirror_parent 64 64 2021-03-09T02:49:43.451 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:pool 'mirror_parent' already exists 2021-03-09T02:49:43.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster2 pool init mirror 2021-03-09T02:49:46.341 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster2 pool init mirror_parent 2021-03-09T02:49:49.375 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:49.375 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster2 mirror pool enable --site-name cluster2 mirror image 2021-03-09T02:49:49.419 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror pool enable mirror_parent image 2021-03-09T02:49:49.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 namespace create mirror/ns1 2021-03-09T02:49:49.503 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 namespace create mirror/ns2 2021-03-09T02:49:49.547 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns1 image 2021-03-09T02:49:49.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror pool enable mirror/ns2 image 2021-03-09T02:49:49.643 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -z ] 2021-03-09T02:49:49.643 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -z ] 2021-03-09T02:49:49.643 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ peer_add cluster2 mirror cluster1 2021-03-09T02:49:49.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:49:49.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:49:49.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.645 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local client_cluster=cluster1 2021-03-09T02:49:49.645 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.645 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local remote_cluster=cluster1 2021-03-09T02:49:49.645 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local uuid_var_name 2021-03-09T02:49:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local error_code 2021-03-09T02:49:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local peer_uuid 2021-03-09T02:49:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:49:49.646 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror pool peer add mirror cluster1 2021-03-09T02:49:49.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ peer_uuid=e763cae0-ac93-49cb-ba80-7dcb606742ec 2021-03-09T02:49:49.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=0 2021-03-09T02:49:49.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:49:49.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 -eq 17 ] 2021-03-09T02:49:49.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 0 -eq 0 2021-03-09T02:49:49.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:49.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:49:49.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ peer_add cluster2 mirror_parent cluster1 2021-03-09T02:49:49.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:49:49.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror_parent 2021-03-09T02:49:49.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local client_cluster=cluster1 2021-03-09T02:49:49.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local remote_cluster=cluster1 2021-03-09T02:49:49.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local uuid_var_name 2021-03-09T02:49:49.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:49.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local error_code 2021-03-09T02:49:49.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local peer_uuid 2021-03-09T02:49:49.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:49:49.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror pool peer add mirror_parent cluster1 2021-03-09T02:49:49.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ peer_uuid=2fe9f24b-e476-4019-b959-47145cf3b670 2021-03-09T02:49:49.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=0 2021-03-09T02:49:49.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:49:49.733 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 -eq 17 ] 2021-03-09T02:49:49.733 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 0 -eq 0 2021-03-09T02:49:49.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:49.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:49:49.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:49:49.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ start_mirrors cluster1 2021-03-09T02:49:49.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:49.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:49:49.748 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ start_mirror cluster1:0 2021-03-09T02:49:49.748 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:49:49.748 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:49.748 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:49:49.749 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:49:49.749 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:49.749 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:49.750 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:49.750 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:49.750 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:49:49.750 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:49.751 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:49.751 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:49:49.751 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:49.751 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 1 2021-03-09T02:49:49.751 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 2021-03-09T02:49:49.752 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ start_mirror cluster1:1 2021-03-09T02:49:49.752 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:49:49.752 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:49.752 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:49:49.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:49:49.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:49.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:49.754 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:49.754 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:49.754 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:49:49.755 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:49.755 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:49.755 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:49:49.755 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:49.756 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 1 2021-03-09T02:49:49.756 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 2021-03-09T02:49:49.756 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ start_mirror cluster1:2 2021-03-09T02:49:49.756 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:2 2021-03-09T02:49:49.757 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:49.757 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-03-09T02:49:49.757 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:2 2021-03-09T02:49:49.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:49.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:49.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:49.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:49.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 2 = cluster1:2 ] 2021-03-09T02:49:49.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:49.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:49.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=2 2021-03-09T02:49:49.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:49.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 1 2021-03-09T02:49:49.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 2021-03-09T02:49:49.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ start_mirror cluster1:3 2021-03-09T02:49:49.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:3 2021-03-09T02:49:49.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:49.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-03-09T02:49:49.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:3 2021-03-09T02:49:49.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:49.762 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:49.762 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:49.762 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:49.763 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 3 = cluster1:3 ] 2021-03-09T02:49:49.763 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:49.763 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:49.763 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=3 2021-03-09T02:49:49.763 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:49.764 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 1 2021-03-09T02:49:49.764 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 2021-03-09T02:49:49.764 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ start_mirrors cluster2 2021-03-09T02:49:49.764 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:49:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:49:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ start_mirror cluster2:0 2021-03-09T02:49:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2:0 2021-03-09T02:49:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster2:0 cluster instance 2021-03-09T02:49:49.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster2:0 2021-03-09T02:49:49.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:49.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:49.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster2 2021-03-09T02:49:49.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:49.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster2:0 ] 2021-03-09T02:49:49.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster2 2021-03-09T02:49:49.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster2 2021-03-09T02:49:49.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:49:49.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:49.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 1 2021-03-09T02:49:49.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 2021-03-09T02:49:49.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ start_mirror cluster2:1 2021-03-09T02:49:49.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2:1 2021-03-09T02:49:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster2:1 cluster instance 2021-03-09T02:49:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster2:1 2021-03-09T02:49:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:49.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster2 2021-03-09T02:49:49.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:49.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster2:1 ] 2021-03-09T02:49:49.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster2 2021-03-09T02:49:49.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster2 2021-03-09T02:49:49.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:49:49.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:49.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 1 2021-03-09T02:49:49.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 2021-03-09T02:49:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ start_mirror cluster2:2 2021-03-09T02:49:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2:2 2021-03-09T02:49:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster2:2 cluster instance 2021-03-09T02:49:49.773 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster2:2 2021-03-09T02:49:49.773 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:49.773 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:49.773 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster2 2021-03-09T02:49:49.774 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:49.774 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 2 = cluster2:2 ] 2021-03-09T02:49:49.774 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster2 2021-03-09T02:49:49.774 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster2 2021-03-09T02:49:49.774 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=2 2021-03-09T02:49:49.775 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:49.775 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 1 2021-03-09T02:49:49.775 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 2021-03-09T02:49:49.775 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ start_mirror cluster2:3 2021-03-09T02:49:49.775 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2:3 2021-03-09T02:49:49.776 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:49.776 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster2:3 cluster instance 2021-03-09T02:49:49.776 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster2:3 2021-03-09T02:49:49.776 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:49.777 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:49.777 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster2 2021-03-09T02:49:49.777 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:49.777 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 3 = cluster2:3 ] 2021-03-09T02:49:49.777 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster2 2021-03-09T02:49:49.778 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster2 2021-03-09T02:49:49.778 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=3 2021-03-09T02:49:49.778 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:49.778 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 1 2021-03-09T02:49:49.778 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 2021-03-09T02:49:49.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ testlog TEST: add image and test replay after client crashes 2021-03-09T02:49:49.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + tee -a /tmp/tmp.y6RrY8NxKl/rbd-mirror.test.log 2021-03-09T02:49:49.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:date +%F %T 2021-03-09T02:49:49.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo 2021-03-09 02:49:49 TEST: add image and test replay after client crashes 2021-03-09T02:49:49.780 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:2021-03-09 02:49:49 TEST: add image and test replay after client crashes 2021-03-09T02:49:49.780 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=test 2021-03-09T02:49:49.780 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror test snapshot 512M 2021-03-09T02:49:49.780 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:49:49.780 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.781 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:49:49.781 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.781 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:49:49.781 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.781 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T02:49:49.782 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T02:49:49.782 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.782 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror test 512M 2021-03-09T02:49:49.782 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:49:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:49:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:49:49.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.784 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T02:49:49.784 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 512M ] 2021-03-09T02:49:49.784 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=512M 2021-03-09T02:49:49.784 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:49.784 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 512M --image-feature layering,exclusive-lock mirror/test 2021-03-09T02:49:49.796 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T02:49:49.796 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror test snapshot 2021-03-09T02:49:49.797 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:49:49.797 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:49:49.797 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:49:49.797 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T02:49:49.797 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/test snapshot 2021-03-09T02:49:50.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T02:49:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:49:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:49:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:49:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:49:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:50.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:49:50.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:49:50.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:49:50.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:49:50.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:49:51.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:49:51.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:51.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:49:51.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:49:51.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:49:51.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:49:51.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:49:51.450 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:49:51.451 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:49:51.451 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:51.451 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:51.451 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance=cluster1 2021-03-09T02:49:51.452 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loop_instance 2021-03-09T02:49:51.452 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + sleep 0 2021-03-09T02:49:51.452 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:grep -i state 2021-03-09T02:49:51.452 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:49:51.453 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:49:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:49:51.455 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:49:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:49:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:49:51.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:49:51.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:49:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:51.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:49:51.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:49:51.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:51.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:51.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:49:51.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:49:51.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:51.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:51.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:51.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:49:51.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:51.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:51.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:49:51.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:51.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:49:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:51.474 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:51.475 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:51.475 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:49:51.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:51.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:51.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:51.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:49:51.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:49:51.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:51.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:51.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:49:51.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:49:51.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:51.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:51.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:51.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:49:51.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:51.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:49:51.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:51.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:49:51.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:49:51.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:51.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:51.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:49:51.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:49:51.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:51.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:51.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:51.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:49:51.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:51.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:51.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:49:51.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:51.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:49:51.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:49:51.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:49:51.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:49:51.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:49:51.701 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:51.701 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:51.701 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:51.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-03-09T02:49:51.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:2 2021-03-09T02:49:51.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:51.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:51.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-03-09T02:49:51.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:2 2021-03-09T02:49:51.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:51.708 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:51.708 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.708 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:51.708 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 2 = cluster1:2 ] 2021-03-09T02:49:51.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:51.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=2 2021-03-09T02:49:51.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:51.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-03-09T02:49:51.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:2 2021-03-09T02:49:51.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:51.711 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:51.711 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-03-09T02:49:51.711 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:2 2021-03-09T02:49:51.711 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 2 = cluster1:2 ] 2021-03-09T02:49:51.712 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:51.713 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:51.713 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=2 2021-03-09T02:49:51.713 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:51.713 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-03-09T02:49:51.723 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-03-09T02:49:51.724 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-03-09T02:49:51.724 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-03-09T02:49:51.724 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-03-09T02:49:51.817 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:51.817 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:51.818 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:51.820 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-03-09T02:49:51.820 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:3 2021-03-09T02:49:51.820 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:51.821 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:51.821 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-03-09T02:49:51.821 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:3 2021-03-09T02:49:51.821 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:51.822 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:51.822 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.822 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:51.822 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 3 = cluster1:3 ] 2021-03-09T02:49:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=3 2021-03-09T02:49:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:51.823 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-03-09T02:49:51.824 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:3 2021-03-09T02:49:51.824 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:51.824 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:51.824 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-03-09T02:49:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:3 2021-03-09T02:49:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:51.825 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:51.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 3 = cluster1:3 ] 2021-03-09T02:49:51.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:51.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:51.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=3 2021-03-09T02:49:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:51.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-03-09T02:49:51.838 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-03-09T02:49:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-03-09T02:49:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-03-09T02:49:51.839 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-03-09T02:49:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:51.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:51.936 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:49:52.937 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:49:52.937 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:49:52.940 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:49:52.940 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:49:52.940 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:52.941 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:52.941 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:49:52.941 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:49:52.941 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:52.941 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:52.942 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:52.942 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:52.942 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:49:52.942 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:52.943 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:52.943 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:49:52.943 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:52.943 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:49:52.943 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:49:52.944 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:52.944 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:52.944 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:49:52.944 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:49:52.945 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:52.945 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:52.945 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:52.945 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:52.945 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:49:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:49:52.946 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:52.947 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:49:52.962 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:52.962 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:52.962 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:52.962 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:49:53.067 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:53.067 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:53.067 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:53.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:49:53.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:49:53.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:53.072 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:53.072 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:49:53.072 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:49:53.073 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:53.074 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:53.074 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:53.075 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:53.075 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:49:53.075 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:53.076 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:53.076 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:49:53.076 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:53.076 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:49:53.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:49:53.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:53.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:53.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:49:53.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:49:53.078 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:53.078 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:53.078 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:53.078 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:53.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:49:53.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:53.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:53.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:49:53.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:53.080 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:49:53.089 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:49:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:49:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:49:53.090 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:49:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:53.190 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-03-09T02:49:53.193 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:2 2021-03-09T02:49:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-03-09T02:49:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:2 2021-03-09T02:49:53.194 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:53.195 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 2 = cluster1:2 ] 2021-03-09T02:49:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=2 2021-03-09T02:49:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:53.196 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-03-09T02:49:53.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:2 2021-03-09T02:49:53.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:53.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:53.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-03-09T02:49:53.198 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:2 2021-03-09T02:49:53.198 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:53.198 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:53.198 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:53.198 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:53.199 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 2 = cluster1:2 ] 2021-03-09T02:49:53.199 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:53.199 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:53.199 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=2 2021-03-09T02:49:53.200 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:53.200 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-03-09T02:49:53.213 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-03-09T02:49:53.214 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-03-09T02:49:53.214 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-03-09T02:49:53.214 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-03-09T02:49:53.308 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:53.309 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:53.309 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:53.318 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-03-09T02:49:53.318 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:3 2021-03-09T02:49:53.318 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:53.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:53.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-03-09T02:49:53.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:3 2021-03-09T02:49:53.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 3 = cluster1:3 ] 2021-03-09T02:49:53.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=3 2021-03-09T02:49:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:53.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-03-09T02:49:53.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:3 2021-03-09T02:49:53.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:53.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-03-09T02:49:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:3 2021-03-09T02:49:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:53.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:53.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:53.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 3 = cluster1:3 ] 2021-03-09T02:49:53.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:53.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:53.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=3 2021-03-09T02:49:53.325 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:53.325 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-03-09T02:49:53.329 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-03-09T02:49:53.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-03-09T02:49:53.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-03-09T02:49:53.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-03-09T02:49:53.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:53.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:53.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:53.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 2 2021-03-09T02:49:55.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:49:55.427 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:49:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:49:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:49:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:55.429 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:49:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:49:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:55.430 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:55.431 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:49:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:49:55.432 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:55.433 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:49:55.434 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:49:55.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:55.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:55.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:49:55.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:49:55.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:55.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:55.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:55.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:49:55.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:55.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:55.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:49:55.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:55.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:49:55.453 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:55.453 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:55.453 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:55.453 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:49:55.554 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:55.555 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:55.555 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:55.557 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:49:55.557 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:49:55.558 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:55.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:55.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:49:55.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:49:55.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:55.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:55.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:55.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:49:55.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:55.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:49:55.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:55.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:49:55.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:49:55.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:55.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:55.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:49:55.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:49:55.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:55.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:55.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:55.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:49:55.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:55.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:55.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:49:55.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:49:55.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:49:55.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:49:55.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:49:55.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:49:55.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:49:55.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:55.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:55.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:55.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:2 rbd mirror status mirror/test 2021-03-09T02:49:55.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:2 2021-03-09T02:49:55.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:55.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:55.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:2 cluster instance 2021-03-09T02:49:55.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:2 2021-03-09T02:49:55.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:55.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:55.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:55.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 2 = cluster1:2 ] 2021-03-09T02:49:55.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:55.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=2 2021-03-09T02:49:55.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:55.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:2 cluster1 2021-03-09T02:49:55.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:2 2021-03-09T02:49:55.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:55.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:55.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:2 local_cluster instance 2021-03-09T02:49:55.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:2 2021-03-09T02:49:55.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:55.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:55.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:55.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 2 = cluster1:2 ] 2021-03-09T02:49:55.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:55.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:55.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=2 2021-03-09T02:49:55.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=2 2021-03-09T02:49:55.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.2 admin socket 2021-03-09T02:49:55.703 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-03-09T02:49:55.703 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-03-09T02:49:55.705 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok 2021-03-09T02:49:55.705 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.2.asok rbd mirror status mirror/test 2021-03-09T02:49:55.797 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:55.797 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:55.797 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:55.799 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:3 rbd mirror status mirror/test 2021-03-09T02:49:55.800 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:3 2021-03-09T02:49:55.800 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:55.800 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:55.801 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:3 cluster instance 2021-03-09T02:49:55.801 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:3 2021-03-09T02:49:55.802 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:55.802 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:55.802 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.802 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:55.802 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 3 = cluster1:3 ] 2021-03-09T02:49:55.803 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:55.803 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.803 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=3 2021-03-09T02:49:55.803 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:55.804 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:3 cluster1 2021-03-09T02:49:55.804 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:3 2021-03-09T02:49:55.804 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:55.805 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:55.805 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:3 local_cluster instance 2021-03-09T02:49:55.805 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:3 2021-03-09T02:49:55.805 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:55.805 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 3 = cluster1:3 ] 2021-03-09T02:49:55.806 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:55.807 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:55.807 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=3 2021-03-09T02:49:55.807 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=3 2021-03-09T02:49:55.807 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.3 admin socket 2021-03-09T02:49:55.820 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-03-09T02:49:55.820 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-03-09T02:49:55.821 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok 2021-03-09T02:49:55.821 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.3.asok rbd mirror status mirror/test 2021-03-09T02:49:55.914 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:49:55.915 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:49:55.915 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:49:55.916 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 4 2021-03-09T02:49:59.917 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:49:59.917 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:49:59.919 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:49:59.920 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:49:59.920 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:49:59.920 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:59.921 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:49:59.921 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:49:59.922 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:49:59.922 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:59.922 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:59.922 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:59.923 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:49:59.923 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:49:59.923 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:59.923 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:49:59.923 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:49:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:49:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:49:59.924 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:49:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:49:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:49:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:49:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:49:59.925 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:49:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:49:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:49:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:49:59.926 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:49:59.927 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:49:59.927 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:49:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:59.939 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:59.940 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:49:59.940 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:50:00.049 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:50:00.049 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:50:00.050 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:50:00.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:50:00.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:50:00.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:50:00.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:50:00.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:50:00.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:50:00.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:50:00.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:50:00.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:50:00.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:50:00.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:50:00.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:50:00.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:50:00.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:50:00.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:50:00.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:50:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:50:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:50:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:50:00.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:50:00.061 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:50:00.061 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:50:00.061 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:50:00.062 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:50:00.062 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:50:00.062 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:50:00.063 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:50:00.063 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:50:00.063 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:50:00.063 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:50:00.064 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:50:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:50:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:50:00.065 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:50:00.171 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:50:00.172 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:50:00.172 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo "state": "Replaying" 2021-03-09T02:50:00.172 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep -i Replaying 2021-03-09T02:50:00.173 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:50:00.176 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:50:00.177 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:50:00.177 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:50:00.177 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ clean_snap_name= 2021-03-09T02:50:00.177 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 1 10 2021-03-09T02:50:00.178 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror test 2021-03-09T02:50:00.178 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:50:00.178 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:00.179 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:00.179 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T02:50:00.179 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=9 2021-03-09T02:50:00.179 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:50:00.180 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-03-09T02:50:00.308 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 3726336 bytes 2021-03-09T02:50:00.412 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 21959680 bytes 2021-03-09T02:50:00.541 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 53093376 bytes 2021-03-09T02:50:00.653 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 70265856 bytes 2021-03-09T02:50:00.901 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 167860224 bytes 2021-03-09T02:50:01.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 215585792 bytes 2021-03-09T02:50:01.231 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 253685760 bytes 2021-03-09T02:50:01.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 344630272 bytes 2021-03-09T02:50:01.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 345961472 bytes 2021-03-09T02:50:01.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 364238848 bytes 2021-03-09T02:50:01.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 376347648 bytes 2021-03-09T02:50:01.922 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 430590976 bytes 2021-03-09T02:50:02.100 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 483778560 bytes 2021-03-09T02:50:02.173 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 492021760 bytes 2021-03-09T02:50:02.364 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 548537344 bytes 2021-03-09T02:50:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 581061632 bytes 2021-03-09T02:50:02.728 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 671219712 bytes 2021-03-09T02:50:02.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 698840064 bytes 2021-03-09T02:50:02.975 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 706562048 bytes 2021-03-09T02:50:03.045 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 712188928 bytes 2021-03-09T02:50:03.199 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 749168640 bytes 2021-03-09T02:50:03.486 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 851937280 bytes 2021-03-09T02:50:03.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 864838656 bytes 2021-03-09T02:50:03.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 891924480 bytes 2021-03-09T02:50:03.812 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 922717184 bytes 2021-03-09T02:50:03.974 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 980011008 bytes 2021-03-09T02:50:04.081 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1000858624 bytes 2021-03-09T02:50:04.236 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1037817856 bytes 2021-03-09T02:50:04.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1076113408 bytes 2021-03-09T02:50:04.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1149498368 bytes 2021-03-09T02:50:04.847 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1224187904 bytes 2021-03-09T02:50:05.036 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1273559040 bytes 2021-03-09T02:50:05.144 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1291365376 bytes 2021-03-09T02:50:05.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1314244608 bytes 2021-03-09T02:50:05.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1355314176 bytes 2021-03-09T02:50:05.668 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1417053184 bytes 2021-03-09T02:50:05.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1421053952 bytes 2021-03-09T02:50:05.938 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1478926336 bytes 2021-03-09T02:50:05.995 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T02:50:05.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T02:50:05.997 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T02:50:05.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T02:50:05.994+0000 7ff2ddf26700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 20048) UID: 1000 2021-03-09T02:50:06.098 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1508422656 bytes 2021-03-09T02:50:06.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T02:50:06.216 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4000 IOs, 1531400192 bytes 2021-03-09T02:50:06.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4100 IOs, 1593454592 bytes 2021-03-09T02:50:06.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4200 IOs, 1642995712 bytes 2021-03-09T02:50:06.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4300 IOs, 1664604160 bytes 2021-03-09T02:50:07.014 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4400 IOs, 1749214208 bytes 2021-03-09T02:50:07.236 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4500 IOs, 1813912576 bytes 2021-03-09T02:50:07.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4600 IOs, 1864410112 bytes 2021-03-09T02:50:07.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4700 IOs, 1928253440 bytes 2021-03-09T02:50:07.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4800 IOs, 1966629888 bytes 2021-03-09T02:50:08.006 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4900 IOs, 2047832064 bytes 2021-03-09T02:50:08.102 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5000 IOs, 2060238848 bytes 2021-03-09T02:50:08.269 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5100 IOs, 2107103232 bytes 2021-03-09T02:50:08.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5200 IOs, 2128520192 bytes 2021-03-09T02:50:08.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5300 IOs, 2139159552 bytes 2021-03-09T02:50:08.642 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5400 IOs, 2178786304 bytes 2021-03-09T02:50:08.750 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5500 IOs, 2214691840 bytes 2021-03-09T02:50:08.919 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5600 IOs, 2240783360 bytes 2021-03-09T02:50:09.021 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5700 IOs, 2285626368 bytes 2021-03-09T02:50:09.179 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T02:50:09.179 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:50:09.179 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T02:50:09.179 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:50:09.180 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:50:09.180 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:09.180 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:09.180 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:09.180 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:50:09.181 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:50:09.181 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:50:09.181 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:50:09.998 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T02:50:09.998 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T02:50:09.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:50:09.999 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T02:50:09.999 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T02:50:09.999 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T02:50:10.002 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T02:50:10.002 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T02:50:10.180 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:50:10.181 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:10.181 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:10.181 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:10.181 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:50:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:50:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:50:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-03-09T02:50:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.mirror_status 2021-03-09T02:50:10.182 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:50:10.184 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:50:10.184 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-03-09T02:50:10.184 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ tee /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:50:10.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:test: 2021-03-09T02:50:10.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: global_id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:50:10.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+replaying 2021-03-09T02:50:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"remote_snapshot_timestamp":1615258190,"replay_state":"idle"} 2021-03-09T02:50:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: service: mirror.1 on smithi144 2021-03-09T02:50:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:50:00 2021-03-09T02:50:10.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: peer_sites: 2021-03-09T02:50:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: name: cluster2 2021-03-09T02:50:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+stopped 2021-03-09T02:50:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: local image is primary 2021-03-09T02:50:10.252 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:50:00 2021-03-09T02:50:10.257 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:50:10.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: state: up+replaying 2021-03-09T02:50:10.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ description: .* /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:50:10.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":0.0,"remote_snapshot_timestamp":1615258190,"replay_state":"idle"} 2021-03-09T02:50:10.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:50:10.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo+ up+replaying 2021-03-09T02:50:10.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:grep ^up+ 2021-03-09T02:50:10.263 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:50:10.263 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep service: *mirror..* on /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:50:10.264 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: service: mirror.1 on smithi144 2021-03-09T02:50:10.265 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:50:10.268 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update=2021-03-09 02:50:00 2021-03-09T02:50:10.268 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-03-09 02:50:00 2021-03-09T02:50:10.268 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:10.268 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:10.269 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:10.270 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:50:10.270 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local prev_last_update=2021-03-09 02:50:00 2021-03-09T02:50:10.270 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror.mirror_status 2021-03-09T02:50:10.270 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror.mirror_status 2021-03-09T02:50:10.271 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:50:10.271 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror.mirror_status 2021-03-09T02:50:10.272 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-03-09T02:50:10.310 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd: failed to query services: (13) Permission denied 2021-03-09T02:50:10.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update state 2021-03-09T02:50:10.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-03-09T02:50:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ last_update=2021-03-09 02:50:00 2021-03-09T02:50:10.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-03-09T02:50:10.378 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ state=up+replaying 2021-03-09T02:50:10.379 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:50:10.379 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep up+replaying 2021-03-09T02:50:10.380 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:50:10.382 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:50:10.382 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:50:10.382 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap1 2021-03-09T02:50:10.383 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap1 2021-03-09T02:50:10.383 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:50:10.383 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:10.383 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap1 2021-03-09T02:50:10.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1 --debug-rbd=20 --debug-journaler=20 2021-03-09T02:50:23.995 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.40409027357055705) > weight (0.25) 2021-03-09T02:50:23.996 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9039843480841278) > weight (0.25) 2021-03-09T02:50:23.996 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T02:50:23.997 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T02:50:23.997 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T02:50:23.998 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T02:50:23.998+0000 7f11b2bac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 20108) UID: 1000 2021-03-09T02:50:24.198 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T02:50:28.998 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T02:50:28.998 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T02:50:28.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:50:28.999 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T02:50:28.999 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T02:50:28.999 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T02:50:29.002 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T02:50:29.002 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T02:50:40.249 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:50:40.249 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:40.249 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:40.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:40.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:50:40.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:40.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:40.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:40.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:50:40.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:50:40.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:50:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:50:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:41.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:50:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:50:41.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:50:41.252 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:50:41.253 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:50:41.253 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + shift 2021-03-09T02:50:41.253 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:41.253 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance=cluster1 2021-03-09T02:50:41.254 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ localgrep loop_instance 2021-03-09T02:50:41.254 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: -i state 2021-03-09T02:50:41.254 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0 2021-03-09T02:50:41.254 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:50:41.254 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:50:41.256 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:50:41.257 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:50:41.257 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:50:41.257 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:50:41.257 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:50:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:50:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:50:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:50:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:50:41.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:50:41.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:50:41.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:50:41.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:50:41.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:50:41.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:50:41.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:50:41.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:50:41.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:41.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:50:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:50:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:50:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:50:41.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:50:41.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:50:41.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:50:41.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:50:41.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:50:41.263 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:50:41.263 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:50:41.263 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:50:41.263 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:50:41.275 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:50:41.275 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:50:41.276 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:50:41.276 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:50:41.381 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:50:41.381 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:50:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:50:41.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:50:41.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:50:41.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:50:41.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:50:41.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:50:41.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:50:41.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:50:41.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:50:41.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:50:41.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:50:41.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:50:41.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:50:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:50:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:50:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:50:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:50:41.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:50:41.391 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:41.391 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:50:41.391 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:50:41.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:50:41.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:50:41.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:50:41.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:50:41.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:50:41.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:50:41.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:50:41.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:50:41.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:50:41.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:50:41.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:50:41.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:50:41.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:50:41.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:50:41.406 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:50:41.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:50:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:50:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo "state": "Replaying" 2021-03-09T02:50:41.517 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep -i Replaying 2021-03-09T02:50:41.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:50:41.520 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:50:41.520 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:50:41.520 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:50:41.521 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-03-09T02:50:41.521 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:50:41.521 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:50:41.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:41.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:41.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = journal ] 2021-03-09T02:50:41.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = snapshot ] 2021-03-09T02:50:41.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-03-09T02:50:41.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:50:41.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:50:41.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:41.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:41.524 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster2-mirror-test.status 2021-03-09T02:50:41.524 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster2-mirror-test.status 2021-03-09T02:50:41.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:50:41.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:50:41.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.status 2021-03-09T02:50:41.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.status 2021-03-09T02:50:41.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:50:41.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:50:41.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-03-09T02:50:41.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:50:41.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:41.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:41.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-03-09T02:50:42.279 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Snapshot ID: 6 2021-03-09T02:50:42.290 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:50:42.290 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:50:42.290 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:50:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-03-09T02:50:42.291 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:50:42.340 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-03-09T02:50:42.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snapshot_id=6 2021-03-09T02:50:42.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:50:42.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.2 2021-03-09T02:50:42.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:50:42.546 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:42.546 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:42.546 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:42.546 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:50:42.546 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:50:42.547 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:50:42.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:50:42.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=4 2021-03-09T02:50:42.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 6 = 4 2021-03-09T02:50:42.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.4 2021-03-09T02:50:43.015 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:50:43.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:43.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:43.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:43.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:50:43.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:50:43.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:50:43.083 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:50:43.088 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=4 2021-03-09T02:50:43.088 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 6 = 4 2021-03-09T02:50:43.088 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.8 2021-03-09T02:50:43.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:50:43.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:43.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:43.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:43.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:50:43.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:50:43.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:50:43.953 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:50:43.953 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:50:43.958 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:50:43.958 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:50:43.959 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:50:43.959 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:50:43.959 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 6 = 2021-03-09T02:50:43.959 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1.6 2021-03-09T02:50:45.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:50:45.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:45.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:45.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:45.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:50:45.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:50:45.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:50:45.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:50:45.624 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=6 2021-03-09T02:50:45.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 6 = 6 2021-03-09T02:50:45.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:50:45.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snap_present cluster1 mirror test snap1 2021-03-09T02:50:45.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:50:45.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:45.626 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:45.626 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap1 2021-03-09T02:50:45.626 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:50:45.626 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:50:46.626 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 info mirror/test@snap1 2021-03-09T02:50:46.680 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:rbd image 'test': 2021-03-09T02:50:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: size 512 MiB in 128 objects 2021-03-09T02:50:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: order 22 (4 MiB objects) 2021-03-09T02:50:46.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: snapshot_count: 3 2021-03-09T02:50:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: id: 10a0f27e4330 2021-03-09T02:50:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: block_name_prefix: rbd_data.10a0f27e4330 2021-03-09T02:50:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: format: 2 2021-03-09T02:50:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: features: layering, exclusive-lock, operations, non-primary 2021-03-09T02:50:46.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: op_features: snap-trash 2021-03-09T02:50:46.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: flags: 2021-03-09T02:50:46.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: create_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:50:46.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: access_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:50:46.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: modify_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:50:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: protected: False 2021-03-09T02:50:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring state: enabled 2021-03-09T02:50:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring mode: snapshot 2021-03-09T02:50:46.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring global id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:50:46.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring primary: false 2021-03-09T02:50:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:50:46.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:50:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap1 2021-03-09T02:50:46.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:46.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:46.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap1 2021-03-09T02:50:46.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:50:46.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:50:46.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:50:46.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:50:46.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:50:47.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:50:47.347 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:50:48.162 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:50:48.167 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:50:48.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:50:48.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:50:48.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ clean_snap_name=snap1-clean 2021-03-09T02:50:48.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap1-clean 2021-03-09T02:50:48.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:50:48.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:48.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:48.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap1-clean 2021-03-09T02:50:48.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap1-clean --debug-rbd=20 --debug-journaler=20 2021-03-09T02:50:49.307 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror test 2021-03-09T02:50:49.307 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:50:49.307 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:50:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:50:49.308 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T02:50:49.309 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=12 2021-03-09T02:50:49.309 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:50:49.309 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 12s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-03-09T02:50:49.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 17403904 bytes 2021-03-09T02:50:49.731 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 39211008 bytes 2021-03-09T02:50:49.917 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 83915776 bytes 2021-03-09T02:50:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 105561088 bytes 2021-03-09T02:50:50.192 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 140709888 bytes 2021-03-09T02:50:50.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 165726208 bytes 2021-03-09T02:50:50.413 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 179130368 bytes 2021-03-09T02:50:50.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 204488704 bytes 2021-03-09T02:50:50.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 248428544 bytes 2021-03-09T02:50:50.877 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 291294208 bytes 2021-03-09T02:50:51.073 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 347233280 bytes 2021-03-09T02:50:51.288 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 413312000 bytes 2021-03-09T02:50:51.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 436797440 bytes 2021-03-09T02:50:51.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 489861120 bytes 2021-03-09T02:50:51.672 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 490467328 bytes 2021-03-09T02:50:51.814 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 519785472 bytes 2021-03-09T02:50:52.112 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 615148544 bytes 2021-03-09T02:50:52.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 741731328 bytes 2021-03-09T02:50:52.618 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 778364928 bytes 2021-03-09T02:50:52.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 801319936 bytes 2021-03-09T02:50:52.839 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 818993152 bytes 2021-03-09T02:50:53.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 880887808 bytes 2021-03-09T02:50:53.153 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 895165440 bytes 2021-03-09T02:50:53.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 949787648 bytes 2021-03-09T02:50:53.453 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 968624128 bytes 2021-03-09T02:50:53.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1016207360 bytes 2021-03-09T02:50:53.712 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1022942208 bytes 2021-03-09T02:50:53.791 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1028356096 bytes 2021-03-09T02:50:53.995 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1082782720 bytes 2021-03-09T02:50:54.172 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1158800384 bytes 2021-03-09T02:50:54.413 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1204290560 bytes 2021-03-09T02:50:54.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1233814528 bytes 2021-03-09T02:50:54.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1302899712 bytes 2021-03-09T02:50:54.908 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1326828544 bytes 2021-03-09T02:50:55.108 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1374087168 bytes 2021-03-09T02:50:55.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1447099392 bytes 2021-03-09T02:50:55.419 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1456425984 bytes 2021-03-09T02:50:55.674 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1531207680 bytes 2021-03-09T02:50:55.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1551341568 bytes 2021-03-09T02:50:55.895 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4000 IOs, 1565618176 bytes 2021-03-09T02:50:56.087 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4100 IOs, 1612096512 bytes 2021-03-09T02:50:56.294 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4200 IOs, 1677423616 bytes 2021-03-09T02:50:56.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4300 IOs, 1686128640 bytes 2021-03-09T02:50:56.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4400 IOs, 1708509184 bytes 2021-03-09T02:50:56.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4500 IOs, 1754246144 bytes 2021-03-09T02:50:56.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4600 IOs, 1754842112 bytes 2021-03-09T02:50:56.842 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4700 IOs, 1759554560 bytes 2021-03-09T02:50:57.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4800 IOs, 1816289280 bytes 2021-03-09T02:50:57.155 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4900 IOs, 1849183232 bytes 2021-03-09T02:50:57.326 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5000 IOs, 1866588160 bytes 2021-03-09T02:50:57.510 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5100 IOs, 1909129216 bytes 2021-03-09T02:50:57.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5200 IOs, 1954482176 bytes 2021-03-09T02:50:57.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5300 IOs, 1985999872 bytes 2021-03-09T02:50:57.981 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5400 IOs, 2039214080 bytes 2021-03-09T02:50:58.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5500 IOs, 2071173120 bytes 2021-03-09T02:50:58.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5600 IOs, 2106514432 bytes 2021-03-09T02:50:58.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5700 IOs, 2143214592 bytes 2021-03-09T02:50:58.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5800 IOs, 2194515968 bytes 2021-03-09T02:50:58.951 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5900 IOs, 2278294528 bytes 2021-03-09T02:50:59.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6000 IOs, 2292449280 bytes 2021-03-09T02:50:59.172 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6100 IOs, 2302185472 bytes 2021-03-09T02:50:59.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6200 IOs, 2394853376 bytes 2021-03-09T02:50:59.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6300 IOs, 2419838976 bytes 2021-03-09T02:50:59.867 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6400 IOs, 2505307136 bytes 2021-03-09T02:51:00.083 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6500 IOs, 2559313920 bytes 2021-03-09T02:51:00.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6600 IOs, 2636649472 bytes 2021-03-09T02:51:00.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6700 IOs, 2676405248 bytes 2021-03-09T02:51:00.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6800 IOs, 2698092544 bytes 2021-03-09T02:51:00.840 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6900 IOs, 2717234176 bytes 2021-03-09T02:51:01.120 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:7000 IOs, 2790316032 bytes 2021-03-09T02:51:01.215 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:7100 IOs, 2794832896 bytes 2021-03-09T02:51:01.313 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T02:51:01.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:51:01.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T02:51:01.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:51:01.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:51:01.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:01.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:01.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:01.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:51:01.316 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:51:01.316 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:51:01.316 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:51:02.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:51:02.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:02.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:02.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:02.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:51:02.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:51:02.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:51:02.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-03-09T02:51:02.316 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.mirror_status 2021-03-09T02:51:02.316 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:51:02.317 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:51:02.317 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + tee /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:51:02.317 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-03-09T02:51:02.383 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:test: 2021-03-09T02:51:02.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: global_id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:51:02.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+replaying 2021-03-09T02:51:02.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: replaying, {"bytes_per_second":17757900.8,"bytes_per_snapshot":266368512.0,"local_snapshot_timestamp":1615258242,"remote_snapshot_timestamp":1615258242,"replay_state":"idle"} 2021-03-09T02:51:02.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: service: mirror.1 on smithi144 2021-03-09T02:51:02.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:50:57 2021-03-09T02:51:02.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: peer_sites: 2021-03-09T02:51:02.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: name: cluster2 2021-03-09T02:51:02.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+stopped 2021-03-09T02:51:02.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: local image is primary 2021-03-09T02:51:02.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:50:58 2021-03-09T02:51:02.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:51:02.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: state: up+replaying 2021-03-09T02:51:02.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ description: .* /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:51:02.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: description: replaying, {"bytes_per_second":17757900.8,"bytes_per_snapshot":266368512.0,"local_snapshot_timestamp":1615258242,"remote_snapshot_timestamp":1615258242,"replay_state":"idle"} 2021-03-09T02:51:02.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:51:02.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:51:02.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^up+ 2021-03-09T02:51:02.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:51:02.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep service: *mirror..* on /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:51:02.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: service: mirror.1 on smithi144 2021-03-09T02:51:02.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:51:02.402 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update=2021-03-09 02:50:57 2021-03-09T02:51:02.402 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-03-09 02:50:57 2021-03-09T02:51:02.403 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:02.403 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:02.403 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:02.403 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:51:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local prev_last_update=2021-03-09 02:50:57 2021-03-09T02:51:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror.mirror_status 2021-03-09T02:51:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror.mirror_status 2021-03-09T02:51:02.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:51:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror.mirror_status 2021-03-09T02:51:02.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-03-09T02:51:02.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd: failed to query services: (13) Permission denied 2021-03-09T02:51:02.489 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update state 2021-03-09T02:51:02.489 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-03-09T02:51:02.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ last_update=2021-03-09 02:50:57 2021-03-09T02:51:02.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-03-09T02:51:02.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ state=up+replaying 2021-03-09T02:51:02.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:51:02.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep up+replaying 2021-03-09T02:51:02.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:51:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:51:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:51:02.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap2 2021-03-09T02:51:02.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap2 2021-03-09T02:51:02.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:51:02.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:02.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:02.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap2 2021-03-09T02:51:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2 --debug-rbd=20 --debug-journaler=20 2021-03-09T02:51:32.310 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:51:32.310 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:32.310 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:32.311 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:32.311 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:51:32.311 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:32.311 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:32.311 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:32.312 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:51:32.312 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:51:32.312 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:51:33.311 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:51:33.311 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:33.312 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:33.312 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:33.312 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:51:33.312 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:51:33.313 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:51:33.313 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:51:33.313 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:51:33.313 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:51:33.313 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:33.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + local instance=cluster1 2021-03-09T02:51:33.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loop_instance 2021-03-09T02:51:33.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0 2021-03-09T02:51:33.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:grep -i state 2021-03-09T02:51:33.316 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:51:33.317 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:51:33.317 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:51:33.317 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:51:33.317 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:51:33.318 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:51:33.318 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:51:33.318 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:51:33.318 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:51:33.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:51:33.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:51:33.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:51:33.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:51:33.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:51:33.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:51:33.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:51:33.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:51:33.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:51:33.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:51:33.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:33.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:51:33.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:51:33.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:51:33.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:51:33.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:51:33.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:51:33.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:51:33.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:51:33.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:51:33.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:51:33.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:51:33.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:51:33.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:51:33.335 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:51:33.335 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:51:33.336 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:51:33.336 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:51:33.432 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:51:33.432 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:51:33.432 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:51:33.434 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:51:33.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:51:33.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:51:33.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:51:33.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:51:33.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:51:33.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:51:33.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:51:33.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:51:33.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:51:33.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:51:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:51:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:51:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:51:33.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:51:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:51:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:51:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:51:33.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:51:33.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:51:33.440 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:51:33.440 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:51:33.440 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:51:33.440 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:51:33.440 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:51:33.441 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:51:33.455 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:51:33.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:51:33.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:51:33.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:51:33.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:51:33.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:51:33.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo "state": "Replaying" 2021-03-09T02:51:33.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep -i Replaying 2021-03-09T02:51:33.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:51:33.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:51:33.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:51:33.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:51:33.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-03-09T02:51:33.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:51:33.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:51:33.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:33.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:33.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = journal ] 2021-03-09T02:51:33.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = snapshot ] 2021-03-09T02:51:33.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-03-09T02:51:33.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:51:33.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:51:33.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:33.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:33.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster2-mirror-test.status 2021-03-09T02:51:33.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster2-mirror-test.status 2021-03-09T02:51:33.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:51:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:51:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.status 2021-03-09T02:51:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.status 2021-03-09T02:51:33.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:51:33.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:51:33.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-03-09T02:51:33.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:51:33.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:33.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:33.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-03-09T02:51:34.318 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Snapshot ID: 10 2021-03-09T02:51:34.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:51:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:51:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:51:34.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-03-09T02:51:34.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:51:34.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-03-09T02:51:34.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snapshot_id=10 2021-03-09T02:51:34.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:51:34.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.2 2021-03-09T02:51:34.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:51:34.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:34.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:34.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:34.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:51:34.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:51:34.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:51:34.652 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:51:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=6 2021-03-09T02:51:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 10 = 6 2021-03-09T02:51:34.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.4 2021-03-09T02:51:35.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:51:35.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:35.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:35.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:35.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:51:35.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:51:35.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:51:35.123 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:51:35.123 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:51:35.126 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:51:35.126 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:51:35.127 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:51:35.127 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:51:35.128 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 10 = 2021-03-09T02:51:35.128 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.8 2021-03-09T02:51:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:51:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:35.929 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:35.930 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:51:35.930 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:51:35.930 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:51:35.994 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:51:35.994 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:51:35.999 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:51:35.999 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:51:35.999 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:51:36.000 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:51:36.000 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 10 = 2021-03-09T02:51:36.000 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1.6 2021-03-09T02:51:37.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:51:37.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:37.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:37.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:37.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:51:37.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:51:37.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:51:37.720 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:51:37.721 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:51:37.724 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:51:37.724 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:51:37.724 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:51:37.725 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:51:37.725 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 10 = 2021-03-09T02:51:37.725 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 2 2021-03-09T02:51:39.726 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:51:39.726 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:39.726 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:39.726 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:39.727 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:51:39.727 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:51:39.727 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:51:39.793 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:51:39.799 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=10 2021-03-09T02:51:39.799 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 10 = 10 2021-03-09T02:51:39.800 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:51:39.800 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snap_present cluster1 mirror test snap2 2021-03-09T02:51:39.800 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:39.800 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap2 2021-03-09T02:51:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:51:39.801 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:51:40.800 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 info mirror/test@snap2 2021-03-09T02:51:40.859 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:rbd image 'test': 2021-03-09T02:51:40.859 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: size 512 MiB in 128 objects 2021-03-09T02:51:40.860 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: order 22 (4 MiB objects) 2021-03-09T02:51:40.860 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: snapshot_count: 4 2021-03-09T02:51:40.860 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: id: 10a0f27e4330 2021-03-09T02:51:40.860 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: block_name_prefix: rbd_data.10a0f27e4330 2021-03-09T02:51:40.860 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: format: 2 2021-03-09T02:51:40.861 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: features: layering, exclusive-lock, non-primary 2021-03-09T02:51:40.861 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: op_features: 2021-03-09T02:51:40.861 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: flags: 2021-03-09T02:51:40.861 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: create_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:51:40.862 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: access_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:51:40.862 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: modify_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:51:40.862 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: protected: False 2021-03-09T02:51:40.862 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring state: enabled 2021-03-09T02:51:40.862 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring mode: snapshot 2021-03-09T02:51:40.863 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring global id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:51:40.863 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring primary: false 2021-03-09T02:51:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:51:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snap1-clean ] 2021-03-09T02:51:40.865 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap1-clean 2021-03-09T02:51:40.866 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:40.866 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:40.866 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap1-clean 2021-03-09T02:51:40.866 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:51:40.867 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:51:40.867 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:51:40.867 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:51:40.868 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:51:41.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:51:41.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:51:42.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:51:42.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:51:42.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:51:42.816 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:51:42.817 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap2 2021-03-09T02:51:42.817 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:42.818 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:42.818 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap2 2021-03-09T02:51:42.818 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:51:42.818 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:51:42.819 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:51:42.819 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:51:42.819 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:51:43.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:51:43.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:51:44.308 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:51:44.312 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:51:44.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:51:44.731 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:51:44.731 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ clean_snap_name=snap2-clean 2021-03-09T02:51:44.731 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap2-clean 2021-03-09T02:51:44.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:51:44.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:44.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:44.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap2-clean 2021-03-09T02:51:44.733 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap2-clean --debug-rbd=20 --debug-journaler=20 2021-03-09T02:51:45.348 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror test 2021-03-09T02:51:45.348 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:51:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:45.349 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T02:51:45.350 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=14 2021-03-09T02:51:45.350 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:51:45.350 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-03-09T02:51:45.777 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 94399488 bytes 2021-03-09T02:51:45.989 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 107550720 bytes 2021-03-09T02:51:46.092 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 116724736 bytes 2021-03-09T02:51:46.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 199733248 bytes 2021-03-09T02:51:46.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 204502016 bytes 2021-03-09T02:51:46.740 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 302081024 bytes 2021-03-09T02:51:46.823 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 302692352 bytes 2021-03-09T02:51:46.993 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 339598336 bytes 2021-03-09T02:51:47.148 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 373085184 bytes 2021-03-09T02:51:47.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 415499264 bytes 2021-03-09T02:51:47.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 424199168 bytes 2021-03-09T02:51:47.624 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 472585216 bytes 2021-03-09T02:51:47.720 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 479010816 bytes 2021-03-09T02:51:48.013 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 566498304 bytes 2021-03-09T02:51:48.172 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 598379520 bytes 2021-03-09T02:51:48.366 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 639956992 bytes 2021-03-09T02:51:48.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 690355200 bytes 2021-03-09T02:51:48.775 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 745419776 bytes 2021-03-09T02:51:48.944 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 784817152 bytes 2021-03-09T02:51:49.039 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 806882304 bytes 2021-03-09T02:51:49.185 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 816687104 bytes 2021-03-09T02:51:49.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 892487680 bytes 2021-03-09T02:51:49.668 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 947380224 bytes 2021-03-09T02:51:49.801 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 971992064 bytes 2021-03-09T02:51:49.994 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1040908288 bytes 2021-03-09T02:51:50.201 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1077319680 bytes 2021-03-09T02:51:50.408 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1134808064 bytes 2021-03-09T02:51:50.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1168304128 bytes 2021-03-09T02:51:50.786 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1221425152 bytes 2021-03-09T02:51:50.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1246811136 bytes 2021-03-09T02:51:51.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1336721408 bytes 2021-03-09T02:51:51.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1359264768 bytes 2021-03-09T02:51:51.555 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1388617728 bytes 2021-03-09T02:51:51.748 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1436804096 bytes 2021-03-09T02:51:51.954 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1484228608 bytes 2021-03-09T02:51:52.147 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1530807296 bytes 2021-03-09T02:51:52.281 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1573127168 bytes 2021-03-09T02:51:52.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1660337152 bytes 2021-03-09T02:51:52.815 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1716028416 bytes 2021-03-09T02:51:53.012 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4000 IOs, 1768892416 bytes 2021-03-09T02:51:53.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4100 IOs, 1834375168 bytes 2021-03-09T02:51:53.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4200 IOs, 1845813248 bytes 2021-03-09T02:51:53.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4300 IOs, 1887901696 bytes 2021-03-09T02:51:53.649 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4400 IOs, 1894114304 bytes 2021-03-09T02:51:53.764 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4500 IOs, 1907010560 bytes 2021-03-09T02:51:53.920 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4600 IOs, 1935259648 bytes 2021-03-09T02:51:54.124 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4700 IOs, 1980669952 bytes 2021-03-09T02:51:54.307 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4800 IOs, 2018947072 bytes 2021-03-09T02:51:54.481 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4900 IOs, 2058115072 bytes 2021-03-09T02:51:54.661 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5000 IOs, 2089056256 bytes 2021-03-09T02:51:54.877 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5100 IOs, 2141939712 bytes 2021-03-09T02:51:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5200 IOs, 2175382528 bytes 2021-03-09T02:51:55.299 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5300 IOs, 2233777152 bytes 2021-03-09T02:51:55.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5400 IOs, 2314096640 bytes 2021-03-09T02:51:55.797 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5500 IOs, 2363729920 bytes 2021-03-09T02:51:55.937 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5600 IOs, 2388619264 bytes 2021-03-09T02:51:56.112 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5700 IOs, 2422978560 bytes 2021-03-09T02:51:56.271 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5800 IOs, 2451169280 bytes 2021-03-09T02:51:56.431 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5900 IOs, 2480572416 bytes 2021-03-09T02:51:56.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6000 IOs, 2520375296 bytes 2021-03-09T02:51:56.787 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6100 IOs, 2556338176 bytes 2021-03-09T02:51:56.948 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6200 IOs, 2595774464 bytes 2021-03-09T02:51:57.176 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6300 IOs, 2637154304 bytes 2021-03-09T02:51:57.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6400 IOs, 2681896960 bytes 2021-03-09T02:51:57.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6500 IOs, 2686830592 bytes 2021-03-09T02:51:57.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6600 IOs, 2709578752 bytes 2021-03-09T02:51:57.751 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6700 IOs, 2731871232 bytes 2021-03-09T02:51:58.011 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6800 IOs, 2813580288 bytes 2021-03-09T02:51:58.205 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:6900 IOs, 2854427648 bytes 2021-03-09T02:51:58.377 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:7000 IOs, 2871552000 bytes 2021-03-09T02:51:58.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:7100 IOs, 2941402112 bytes 2021-03-09T02:51:58.751 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:7200 IOs, 2949102592 bytes 2021-03-09T02:51:58.873 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:7300 IOs, 2960102400 bytes 2021-03-09T02:51:59.075 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:7400 IOs, 3012791296 bytes 2021-03-09T02:51:59.204 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:7500 IOs, 3028354048 bytes 2021-03-09T02:51:59.301 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:7600 IOs, 3035129856 bytes 2021-03-09T02:51:59.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T02:51:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:51:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T02:51:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:51:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:51:59.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:51:59.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:51:59.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:51:59.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:51:59.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:51:59.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:51:59.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:52:00.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:52:00.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:00.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:52:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:52:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:52:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-03-09T02:52:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.mirror_status 2021-03-09T02:52:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:52:00.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:52:00.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + CEPH_ARGS= rbd --cluster cluster1 mirrortee image /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status status 2021-03-09T02:52:00.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: mirror/test 2021-03-09T02:52:00.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:test: 2021-03-09T02:52:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: global_id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:52:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+replaying 2021-03-09T02:52:00.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: replaying, {"bytes_per_second":17488486.4,"bytes_per_snapshot":528695808.0,"local_snapshot_timestamp":1615258294,"remote_snapshot_timestamp":1615258294,"replay_state":"idle"} 2021-03-09T02:52:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: service: mirror.1 on smithi144 2021-03-09T02:52:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:51:57 2021-03-09T02:52:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: peer_sites: 2021-03-09T02:52:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: name: cluster2 2021-03-09T02:52:00.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+stopped 2021-03-09T02:52:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: local image is primary 2021-03-09T02:52:00.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:51:58 2021-03-09T02:52:00.433 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:52:00.434 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: state: up+replaying 2021-03-09T02:52:00.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ description: .* /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:52:00.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: description: replaying, {"bytes_per_second":17488486.4,"bytes_per_snapshot":528695808.0,"local_snapshot_timestamp":1615258294,"remote_snapshot_timestamp":1615258294,"replay_state":"idle"} 2021-03-09T02:52:00.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:52:00.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:52:00.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^up+ 2021-03-09T02:52:00.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:52:00.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep service: *mirror..* on /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:52:00.440 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: service: mirror.1 on smithi144 2021-03-09T02:52:00.440 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:52:00.442 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update=2021-03-09 02:51:57 2021-03-09T02:52:00.442 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-03-09 02:51:57 2021-03-09T02:52:00.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:00.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:00.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:00.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:52:00.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local prev_last_update=2021-03-09 02:51:57 2021-03-09T02:52:00.444 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror.mirror_status 2021-03-09T02:52:00.444 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror.mirror_status 2021-03-09T02:52:00.444 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:52:00.445 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror.mirror_status 2021-03-09T02:52:00.445 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-03-09T02:52:00.486 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd: failed to query services: (13) Permission denied 2021-03-09T02:52:00.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update state 2021-03-09T02:52:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-03-09T02:52:00.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ last_update=2021-03-09 02:51:57 2021-03-09T02:52:00.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-03-09T02:52:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ state=up+replaying 2021-03-09T02:52:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:52:00.540 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep up+replaying 2021-03-09T02:52:00.541 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:52:00.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:52:00.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:52:00.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap3 2021-03-09T02:52:00.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap3 2021-03-09T02:52:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:52:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:00.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap3 2021-03-09T02:52:00.546 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3 --debug-rbd=20 --debug-journaler=20 2021-03-09T02:52:04.003 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.610257763166485) > weight (0.25) 2021-03-09T02:52:04.004 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6432066333605664) > weight (0.25) 2021-03-09T02:52:04.004 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.44971723979139044) > weight (0.25) 2021-03-09T02:52:04.004 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T02:52:04.005 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T02:52:04.005 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T02:52:04.006 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T02:52:04.006 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T02:52:04.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T02:52:04.006+0000 7f6e1a5b7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 20070) UID: 1000 2021-03-09T02:52:04.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T02:52:10.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:52:10.109 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T02:52:10.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T02:52:10.110 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T02:52:10.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T02:52:10.114 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T02:52:19.003 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5827227162019452) > weight (0.25) 2021-03-09T02:52:19.003 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T02:52:19.004 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T02:52:19.004 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T02:52:19.005 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T02:52:19.002+0000 7fe05cb13700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 20105) UID: 1000 2021-03-09T02:52:19.206 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T02:52:20.005 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T02:52:20.006 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T02:52:20.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:52:20.007 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T02:52:20.007 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T02:52:20.007 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T02:52:20.010 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T02:52:20.010 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T02:52:30.341 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:52:30.341 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:30.341 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:30.342 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:30.342 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:52:30.342 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:30.342 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:30.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:30.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:52:30.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:52:30.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:52:31.342 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:52:31.342 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:31.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:31.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:31.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:52:31.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:52:31.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:52:31.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:52:31.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:52:31.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:52:31.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:31.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + local instance=cluster1 2021-03-09T02:52:31.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loop_instance 2021-03-09T02:52:31.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0 2021-03-09T02:52:31.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:grep -i state 2021-03-09T02:52:31.348 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:52:31.348 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:52:31.349 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:52:31.349 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:52:31.349 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:52:31.350 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:52:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:52:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:52:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:52:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:52:31.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:52:31.352 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:52:31.352 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:52:31.352 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:52:31.352 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:52:31.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:52:31.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:52:31.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:52:31.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:52:31.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:31.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:52:31.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:52:31.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:52:31.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:52:31.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:52:31.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:52:31.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:52:31.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:52:31.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:52:31.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:52:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:52:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:52:31.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:52:31.368 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:52:31.368 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:52:31.369 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:52:31.370 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:52:31.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:52:31.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:52:31.471 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:52:31.473 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:52:31.473 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:52:31.473 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:52:31.474 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:52:31.474 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:52:31.474 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:52:31.474 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:52:31.475 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:52:31.475 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:52:31.475 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:52:31.475 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:52:31.476 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:52:31.476 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:52:31.476 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:52:31.476 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:52:31.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:52:31.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:52:31.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:31.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:52:31.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:52:31.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:52:31.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:52:31.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:52:31.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:52:31.479 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:52:31.479 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:52:31.479 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:52:31.479 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:52:31.480 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:52:31.480 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:52:31.480 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:52:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:52:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:52:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:52:31.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:52:31.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:52:31.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:52:31.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo "state": "Replaying" 2021-03-09T02:52:31.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep -i Replaying 2021-03-09T02:52:31.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:52:31.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:52:31.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:52:31.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:52:31.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-03-09T02:52:31.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:52:31.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:52:31.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:31.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:31.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = journal ] 2021-03-09T02:52:31.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = snapshot ] 2021-03-09T02:52:31.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-03-09T02:52:31.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:52:31.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:52:31.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:31.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:31.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster2-mirror-test.status 2021-03-09T02:52:31.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster2-mirror-test.status 2021-03-09T02:52:31.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:52:31.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:52:31.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.status 2021-03-09T02:52:31.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.status 2021-03-09T02:52:31.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:52:31.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:52:31.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-03-09T02:52:31.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:52:31.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:31.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:31.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-03-09T02:52:32.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Snapshot ID: 14 2021-03-09T02:52:32.372 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:52:32.372 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:52:32.372 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:32.372 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:32.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:52:32.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-03-09T02:52:32.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:52:32.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-03-09T02:52:32.428 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snapshot_id=14 2021-03-09T02:52:32.429 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:52:32.429 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.2 2021-03-09T02:52:32.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:52:32.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:32.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:32.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:32.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:52:32.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:52:32.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:52:32.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:52:32.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=10 2021-03-09T02:52:32.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 14 = 10 2021-03-09T02:52:32.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.4 2021-03-09T02:52:33.099 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:52:33.099 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:33.099 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:33.099 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:33.100 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:52:33.100 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:52:33.100 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:52:33.163 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:52:33.164 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:52:33.167 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:52:33.168 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:52:33.168 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:52:33.169 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:52:33.169 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 14 = 2021-03-09T02:52:33.169 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.8 2021-03-09T02:52:33.971 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:52:33.971 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:33.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:33.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:52:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:52:33.973 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:52:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:52:34.037 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:52:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:52:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:52:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:52:34.041 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:52:34.042 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 14 = 2021-03-09T02:52:34.042 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1.6 2021-03-09T02:52:35.643 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:52:35.643 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:35.643 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:35.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:35.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:52:35.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:52:35.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:52:35.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:52:35.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:52:35.764 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:52:35.764 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:52:35.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:52:35.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:52:35.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 14 = 2021-03-09T02:52:35.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 2 2021-03-09T02:52:37.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:52:37.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:37.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:37.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:37.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:52:37.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:52:37.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:52:37.837 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:52:37.843 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=14 2021-03-09T02:52:37.843 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 14 = 14 2021-03-09T02:52:37.843 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:52:37.843 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snap_present cluster1 mirror test snap3 2021-03-09T02:52:37.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:37.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:37.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:37.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap3 2021-03-09T02:52:37.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:52:37.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:52:38.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 info mirror/test@snap3 2021-03-09T02:52:38.904 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:rbd image 'test': 2021-03-09T02:52:38.904 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: size 512 MiB in 128 objects 2021-03-09T02:52:38.904 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: order 22 (4 MiB objects) 2021-03-09T02:52:38.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: snapshot_count: 6 2021-03-09T02:52:38.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: id: 10a0f27e4330 2021-03-09T02:52:38.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: block_name_prefix: rbd_data.10a0f27e4330 2021-03-09T02:52:38.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: format: 2 2021-03-09T02:52:38.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: features: layering, exclusive-lock, non-primary 2021-03-09T02:52:38.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: op_features: 2021-03-09T02:52:38.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: flags: 2021-03-09T02:52:38.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: create_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:52:38.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: access_timestamp: Tue Mar 9 02:51:41 2021 2021-03-09T02:52:38.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: modify_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:52:38.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: protected: False 2021-03-09T02:52:38.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring state: enabled 2021-03-09T02:52:38.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring mode: snapshot 2021-03-09T02:52:38.908 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring global id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:52:38.908 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring primary: false 2021-03-09T02:52:38.909 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:52:38.909 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snap2-clean ] 2021-03-09T02:52:38.909 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap2-clean 2021-03-09T02:52:38.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:38.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:38.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap2-clean 2021-03-09T02:52:38.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:52:38.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:52:38.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:52:38.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:52:38.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:52:39.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:52:39.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:52:40.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:52:40.487 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:52:40.764 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:52:40.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:52:40.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap3 2021-03-09T02:52:40.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:40.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:40.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap3 2021-03-09T02:52:40.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:52:40.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:52:40.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:52:40.892 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:52:40.892 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:52:41.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:52:41.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:52:42.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:52:42.444 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:52:42.723 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:52:42.850 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:52:42.851 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ clean_snap_name=snap3-clean 2021-03-09T02:52:42.851 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap3-clean 2021-03-09T02:52:42.851 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:52:42.851 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:42.852 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:42.852 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap3-clean 2021-03-09T02:52:42.852 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap3-clean --debug-rbd=20 --debug-journaler=20 2021-03-09T02:52:43.252 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror test 2021-03-09T02:52:43.252 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:52:43.252 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:43.253 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:43.253 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T02:52:43.254 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=6 2021-03-09T02:52:43.254 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:52:43.255 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-03-09T02:52:43.702 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 79775744 bytes 2021-03-09T02:52:43.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 113661952 bytes 2021-03-09T02:52:44.113 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 152508416 bytes 2021-03-09T02:52:44.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 206883840 bytes 2021-03-09T02:52:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 240565248 bytes 2021-03-09T02:52:44.716 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 273893376 bytes 2021-03-09T02:52:44.873 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 297288704 bytes 2021-03-09T02:52:45.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 331675648 bytes 2021-03-09T02:52:45.207 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 351955968 bytes 2021-03-09T02:52:45.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 373429248 bytes 2021-03-09T02:52:45.481 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 384501760 bytes 2021-03-09T02:52:45.798 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 392445952 bytes 2021-03-09T02:52:45.971 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 423262208 bytes 2021-03-09T02:52:46.088 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 430211072 bytes 2021-03-09T02:52:46.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 430808064 bytes 2021-03-09T02:52:46.350 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 460128256 bytes 2021-03-09T02:52:46.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 494121984 bytes 2021-03-09T02:52:46.748 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 550474752 bytes 2021-03-09T02:52:47.018 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 626819072 bytes 2021-03-09T02:52:47.159 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 645474304 bytes 2021-03-09T02:52:47.402 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 708670464 bytes 2021-03-09T02:52:47.643 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 772665344 bytes 2021-03-09T02:52:47.870 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 830772224 bytes 2021-03-09T02:52:48.080 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 893602816 bytes 2021-03-09T02:52:48.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 958964736 bytes 2021-03-09T02:52:48.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1001422848 bytes 2021-03-09T02:52:48.804 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1045549056 bytes 2021-03-09T02:52:49.096 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1119278080 bytes 2021-03-09T02:52:49.252 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1141795840 bytes 2021-03-09T02:52:49.256 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T02:52:49.257 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:52:49.257 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T02:52:49.257 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:52:49.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:52:49.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:49.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:49.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:49.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:52:49.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:52:49.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:52:49.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:52:50.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:52:50.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:50.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:50.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:50.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:52:50.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:52:50.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:52:50.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-03-09T02:52:50.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.mirror_status 2021-03-09T02:52:50.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:52:50.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:52:50.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-03-09T02:52:50.263 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ tee /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:52:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:test: 2021-03-09T02:52:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: global_id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:52:50.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+replaying 2021-03-09T02:52:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":528695808.0,"local_snapshot_timestamp":1615258294,"remote_snapshot_timestamp":1615258294,"replay_state":"idle"} 2021-03-09T02:52:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: service: mirror.1 on smithi144 2021-03-09T02:52:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:52:27 2021-03-09T02:52:50.333 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: peer_sites: 2021-03-09T02:52:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: name: cluster2 2021-03-09T02:52:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+stopped 2021-03-09T02:52:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: local image is primary 2021-03-09T02:52:50.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:52:28 2021-03-09T02:52:50.340 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:52:50.342 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: state: up+replaying 2021-03-09T02:52:50.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ description: .* /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:52:50.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":528695808.0,"local_snapshot_timestamp":1615258294,"remote_snapshot_timestamp":1615258294,"replay_state":"idle"} 2021-03-09T02:52:50.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:52:50.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + echo up+replaying 2021-03-09T02:52:50.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:grep ^up+ 2021-03-09T02:52:50.346 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:52:50.347 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep service: *mirror..* on /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:52:50.348 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: service: mirror.1 on smithi144 2021-03-09T02:52:50.350 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:52:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update=2021-03-09 02:52:27 2021-03-09T02:52:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-03-09 02:52:27 2021-03-09T02:52:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:52:50.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:52:50.352 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local prev_last_update=2021-03-09 02:52:27 2021-03-09T02:52:50.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror.mirror_status 2021-03-09T02:52:50.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror.mirror_status 2021-03-09T02:52:50.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:52:50.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror.mirror_status 2021-03-09T02:52:50.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-03-09T02:52:50.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd: failed to query services: (13) Permission denied 2021-03-09T02:52:50.446 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update state 2021-03-09T02:52:50.447 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-03-09T02:52:50.452 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ last_update=2021-03-09 02:52:27 2021-03-09T02:52:50.452 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-03-09T02:52:50.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ state=up+replaying 2021-03-09T02:52:50.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:52:50.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep up+replaying 2021-03-09T02:52:50.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:52:50.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:52:50.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:52:50.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap4 2021-03-09T02:52:50.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap4 2021-03-09T02:52:50.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:52:50.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:52:50.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:52:50.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap4 2021-03-09T02:52:50.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4 --debug-rbd=20 --debug-journaler=20 2021-03-09T02:53:16.115 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5363200887223888) > weight (0.25) 2021-03-09T02:53:16.116 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8492384570615642) > weight (0.25) 2021-03-09T02:53:16.116 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T02:53:16.117 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T02:53:16.117 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T02:53:16.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T02:53:16.118+0000 7fae01abe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 20053) UID: 1000 2021-03-09T02:53:16.318 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T02:53:19.118 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T02:53:19.119 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T02:53:19.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:53:19.120 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T02:53:19.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T02:53:19.121 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T02:53:19.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T02:53:19.126 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T02:53:20.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:53:20.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:20.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:20.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:20.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:53:20.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:20.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:20.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:20.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:53:20.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:53:20.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:53:21.321 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:53:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:53:21.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:53:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:53:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:53:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:53:21.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:53:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local+ cluster=cluster1 2021-03-09T02:53:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance=cluster1 2021-03-09T02:53:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ greplocal -i loop_instance state 2021-03-09T02:53:21.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:53:21.325 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0 2021-03-09T02:53:21.327 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:53:21.327 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:53:21.327 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:53:21.328 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:53:21.328 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:53:21.328 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:53:21.328 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:53:21.328 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:53:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:53:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:53:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:53:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:53:21.329 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:53:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:53:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:53:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:53:21.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:53:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:53:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:53:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:53:21.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:53:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:53:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:53:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:53:21.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:53:21.333 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:53:21.333 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:53:21.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:53:21.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:53:21.335 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:53:21.335 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:53:21.335 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:53:21.347 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:53:21.347 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:53:21.348 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:53:21.348 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:53:21.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:53:21.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:53:21.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:53:21.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:53:21.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:53:21.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:53:21.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:53:21.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:53:21.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:53:21.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:53:21.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:53:21.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:53:21.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:53:21.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:53:21.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:53:21.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:53:21.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:53:21.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:53:21.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:53:21.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:53:21.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:21.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:53:21.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:53:21.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:53:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:53:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:53:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:53:21.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:53:21.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:53:21.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:53:21.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:53:21.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:53:21.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:53:21.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:53:21.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:53:21.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:53:21.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:53:21.479 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:53:21.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:53:21.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:53:21.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo "state": "Replaying" 2021-03-09T02:53:21.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep -i Replaying 2021-03-09T02:53:21.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:53:21.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:53:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:53:21.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:53:21.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-03-09T02:53:21.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:53:21.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:53:21.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:21.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:21.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = journal ] 2021-03-09T02:53:21.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = snapshot ] 2021-03-09T02:53:21.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-03-09T02:53:21.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:53:21.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:53:21.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:21.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:21.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster2-mirror-test.status 2021-03-09T02:53:21.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster2-mirror-test.status 2021-03-09T02:53:21.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:53:21.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:53:21.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.status 2021-03-09T02:53:21.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.status 2021-03-09T02:53:21.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:53:21.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:53:21.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-03-09T02:53:21.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:53:21.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:21.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:21.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-03-09T02:53:22.327 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Snapshot ID: 18 2021-03-09T02:53:22.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:53:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:53:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:22.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:53:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-03-09T02:53:22.346 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:53:22.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-03-09T02:53:22.409 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snapshot_id=18 2021-03-09T02:53:22.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:53:22.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.2 2021-03-09T02:53:22.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:53:22.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:22.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:22.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:22.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:53:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:53:22.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:53:22.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:53:22.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=14 2021-03-09T02:53:22.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 18 = 14 2021-03-09T02:53:22.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.4 2021-03-09T02:53:23.086 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:53:23.086 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:53:23.087 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:53:23.088 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:53:23.162 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:53:23.167 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=14 2021-03-09T02:53:23.167 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 18 = 14 2021-03-09T02:53:23.168 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.8 2021-03-09T02:53:23.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:53:23.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:23.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:23.970 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:23.970 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:53:23.970 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:53:23.970 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:53:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:53:24.040 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:53:24.044 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:53:24.045 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:53:24.045 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:53:24.046 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:53:24.046 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 18 = 2021-03-09T02:53:24.047 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1.6 2021-03-09T02:53:25.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:53:25.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:53:25.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:53:25.649 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:53:25.786 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:53:25.786 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:53:25.789 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:53:25.790 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:53:25.790 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:53:25.790 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:53:25.790 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 18 = 2021-03-09T02:53:25.791 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 2 2021-03-09T02:53:27.791 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:53:27.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:27.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:27.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:27.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:53:27.793 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:53:27.793 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:53:27.861 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:53:27.866 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=18 2021-03-09T02:53:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 18 = 18 2021-03-09T02:53:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:53:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snap_present cluster1 mirror test snap4 2021-03-09T02:53:27.867 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap4 2021-03-09T02:53:27.868 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:53:27.869 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:53:28.868 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 info mirror/test@snap4 2021-03-09T02:53:28.934 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:rbd image 'test': 2021-03-09T02:53:28.934 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: size 512 MiB in 128 objects 2021-03-09T02:53:28.934 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: order 22 (4 MiB objects) 2021-03-09T02:53:28.934 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: snapshot_count: 8 2021-03-09T02:53:28.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: id: 10a0f27e4330 2021-03-09T02:53:28.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: block_name_prefix: rbd_data.10a0f27e4330 2021-03-09T02:53:28.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: format: 2 2021-03-09T02:53:28.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: features: layering, exclusive-lock, non-primary 2021-03-09T02:53:28.936 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: op_features: 2021-03-09T02:53:28.936 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: flags: 2021-03-09T02:53:28.936 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: create_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:53:28.936 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: access_timestamp: Tue Mar 9 02:52:42 2021 2021-03-09T02:53:28.936 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: modify_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:53:28.937 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: protected: False 2021-03-09T02:53:28.937 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring state: enabled 2021-03-09T02:53:28.937 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring mode: snapshot 2021-03-09T02:53:28.937 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring global id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:53:28.938 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring primary: false 2021-03-09T02:53:28.939 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:53:28.939 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snap3-clean ] 2021-03-09T02:53:28.940 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap3-clean 2021-03-09T02:53:28.940 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:28.940 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:28.940 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap3-clean 2021-03-09T02:53:28.941 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:53:28.941 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:53:28.941 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:53:28.941 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:53:28.942 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:53:29.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:53:29.634 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:53:30.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:53:30.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:53:30.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:53:30.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:53:30.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap4 2021-03-09T02:53:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap4 2021-03-09T02:53:30.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:53:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:53:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:53:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:53:30.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:53:31.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:53:31.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:53:32.450 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:53:32.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:53:32.730 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:53:32.858 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:53:32.858 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ clean_snap_name=snap4-clean 2021-03-09T02:53:32.859 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap4-clean 2021-03-09T02:53:32.859 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:53:32.859 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:32.859 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:32.860 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap4-clean 2021-03-09T02:53:32.860 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap4-clean --debug-rbd=20 --debug-journaler=20 2021-03-09T02:53:33.274 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror test 2021-03-09T02:53:33.274 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:53:33.275 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:33.275 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:33.275 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T02:53:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=7 2021-03-09T02:53:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:53:33.276 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-03-09T02:53:33.672 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 40151040 bytes 2021-03-09T02:53:33.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 56610816 bytes 2021-03-09T02:53:33.980 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 60404736 bytes 2021-03-09T02:53:34.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 119266304 bytes 2021-03-09T02:53:34.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 157105152 bytes 2021-03-09T02:53:34.638 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 209105920 bytes 2021-03-09T02:53:34.836 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 247732224 bytes 2021-03-09T02:53:35.018 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 279369728 bytes 2021-03-09T02:53:35.116 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 279985152 bytes 2021-03-09T02:53:35.299 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 309729280 bytes 2021-03-09T02:53:35.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 357287936 bytes 2021-03-09T02:53:35.727 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 402039808 bytes 2021-03-09T02:53:36.012 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 481155072 bytes 2021-03-09T02:53:36.143 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 492819456 bytes 2021-03-09T02:53:36.349 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 533943296 bytes 2021-03-09T02:53:36.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 563130368 bytes 2021-03-09T02:53:36.834 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 647120896 bytes 2021-03-09T02:53:37.075 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 722526208 bytes 2021-03-09T02:53:37.302 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 758634496 bytes 2021-03-09T02:53:37.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 778064896 bytes 2021-03-09T02:53:37.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 779862016 bytes 2021-03-09T02:53:37.840 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 851430400 bytes 2021-03-09T02:53:38.196 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 958550016 bytes 2021-03-09T02:53:38.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 982873088 bytes 2021-03-09T02:53:38.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1017445376 bytes 2021-03-09T02:53:38.809 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1085498368 bytes 2021-03-09T02:53:39.096 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1164341248 bytes 2021-03-09T02:53:39.284 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1202115584 bytes 2021-03-09T02:53:39.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1261961216 bytes 2021-03-09T02:53:39.736 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1300678656 bytes 2021-03-09T02:53:39.927 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1335121920 bytes 2021-03-09T02:53:40.043 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1335732224 bytes 2021-03-09T02:53:40.199 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1352728576 bytes 2021-03-09T02:53:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T02:53:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:53:40.283 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T02:53:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:53:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:53:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:40.284 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:40.285 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:40.285 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:53:40.285 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:53:40.285 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:53:40.285 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:53:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:53:41.288 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:41.289 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:41.289 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:41.289 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:53:41.289 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:53:41.290 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:53:41.290 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-03-09T02:53:41.290 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.mirror_status 2021-03-09T02:53:41.291 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:53:41.292 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:53:41.292 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-03-09T02:53:41.292 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ tee /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:53:41.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:test: 2021-03-09T02:53:41.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: global_id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:53:41.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+replaying 2021-03-09T02:53:41.360 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: replaying, {"bytes_per_second":15591082.67,"bytes_per_snapshot":501276672.0,"remote_snapshot_timestamp":1615258402,"replay_state":"syncing","seconds_until_synced":32,"syncing_percent":100,"syncing_snapshot_timestamp":1615258402} 2021-03-09T02:53:41.360 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: service: mirror.1 on smithi144 2021-03-09T02:53:41.360 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:53:27 2021-03-09T02:53:41.360 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: peer_sites: 2021-03-09T02:53:41.361 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: name: cluster2 2021-03-09T02:53:41.361 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+stopped 2021-03-09T02:53:41.361 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: local image is primary 2021-03-09T02:53:41.361 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:53:28 2021-03-09T02:53:41.368 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:53:41.370 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: state: up+replaying 2021-03-09T02:53:41.371 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ description: .* /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:53:41.372 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: description: replaying, {"bytes_per_second":15591082.67,"bytes_per_snapshot":501276672.0,"remote_snapshot_timestamp":1615258402,"replay_state":"syncing","seconds_until_synced":32,"syncing_percent":100,"syncing_snapshot_timestamp":1615258402} 2021-03-09T02:53:41.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:53:41.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:53:41.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^up+ 2021-03-09T02:53:41.374 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:53:41.375 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep service: *mirror..* on /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:53:41.376 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: service: mirror.1 on smithi144 2021-03-09T02:53:41.376 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:53:41.379 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update=2021-03-09 02:53:27 2021-03-09T02:53:41.379 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-03-09 02:53:27 2021-03-09T02:53:41.379 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:53:41.380 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:41.380 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:41.380 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:53:41.380 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local prev_last_update=2021-03-09 02:53:27 2021-03-09T02:53:41.381 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror.mirror_status 2021-03-09T02:53:41.381 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror.mirror_status 2021-03-09T02:53:41.381 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:53:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror.mirror_status 2021-03-09T02:53:41.382 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-03-09T02:53:41.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd: failed to query services: (13) Permission denied 2021-03-09T02:53:41.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update state 2021-03-09T02:53:41.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-03-09T02:53:41.475 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ last_update=2021-03-09 02:53:27 2021-03-09T02:53:41.476 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-03-09T02:53:41.481 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ state=up+replaying 2021-03-09T02:53:41.481 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:53:41.481 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep up+replaying 2021-03-09T02:53:41.482 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:53:41.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:53:41.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:53:41.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap5 2021-03-09T02:53:41.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap5 2021-03-09T02:53:41.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:53:41.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:53:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:53:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap5 2021-03-09T02:53:41.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5 --debug-rbd=20 --debug-journaler=20 2021-03-09T02:53:44.011 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T02:53:44.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T02:53:44.012 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T02:53:44.013 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T02:53:44.014+0000 7f0167bdb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 20088) UID: 1000 2021-03-09T02:53:44.214 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T02:53:51.013 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T02:53:51.014 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T02:53:51.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:53:51.015 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T02:53:51.015 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T02:53:51.015 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T02:53:51.018 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T02:53:51.018 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-03-09T02:54:11.342 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:54:11.342 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:11.342 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:11.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:11.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:54:11.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:11.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:11.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:11.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:54:11.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:54:11.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:54:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:54:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:12.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:12.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:12.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:54:12.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:54:12.346 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:54:12.346 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:54:12.346 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:54:12.346 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:54:12.347 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + local cluster=cluster1 2021-03-09T02:54:12.347 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance=cluster1 2021-03-09T02:54:12.347 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loop_instance 2021-03-09T02:54:12.348 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:grep+ -isleep 0 2021-03-09T02:54:12.348 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state 2021-03-09T02:54:12.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:54:12.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:54:12.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:54:12.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:54:12.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:54:12.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:54:12.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:54:12.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:54:12.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:54:12.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:54:12.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:54:12.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:54:12.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:54:12.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:54:12.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:54:12.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:54:12.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:54:12.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:54:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:54:12.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:54:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:54:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:54:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:54:12.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:54:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:54:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:54:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:54:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:54:12.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:54:12.360 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:54:12.360 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:54:12.360 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:54:12.367 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:54:12.368 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:54:12.368 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:54:12.368 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:54:12.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:54:12.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:54:12.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:54:12.487 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:54:12.488 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:54:12.488 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:54:12.488 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:54:12.488 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:54:12.488 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:54:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:54:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:54:12.491 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:54:12.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:54:12.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:54:12.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:54:12.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:54:12.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:54:12.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:54:12.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:54:12.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:54:12.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:12.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:54:12.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:54:12.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:54:12.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:54:12.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:54:12.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:54:12.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:54:12.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:54:12.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:54:12.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:54:12.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:54:12.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:54:12.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:54:12.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:54:12.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:54:12.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:54:12.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:54:12.616 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:54:12.616 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:54:12.616 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo "state": "Replaying" 2021-03-09T02:54:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep -i Replaying 2021-03-09T02:54:12.617 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:54:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:54:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:54:12.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:54:12.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-03-09T02:54:12.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:54:12.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:54:12.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = journal ] 2021-03-09T02:54:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = snapshot ] 2021-03-09T02:54:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-03-09T02:54:12.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:54:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:54:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster2-mirror-test.status 2021-03-09T02:54:12.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster2-mirror-test.status 2021-03-09T02:54:12.623 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:54:12.624 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:54:12.624 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.status 2021-03-09T02:54:12.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.status 2021-03-09T02:54:12.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:54:12.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:54:12.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-03-09T02:54:12.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:54:12.626 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:12.626 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:12.626 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-03-09T02:54:13.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Snapshot ID: 22 2021-03-09T02:54:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:54:13.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:54:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:54:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-03-09T02:54:13.374 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:54:13.431 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-03-09T02:54:13.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snapshot_id=22 2021-03-09T02:54:13.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:54:13.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.2 2021-03-09T02:54:13.638 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:54:13.638 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:13.639 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:13.639 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:13.639 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:54:13.639 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:54:13.640 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:54:13.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:54:13.714 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=18 2021-03-09T02:54:13.714 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 22 = 18 2021-03-09T02:54:13.715 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.4 2021-03-09T02:54:14.116 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:54:14.117 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:14.117 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:14.117 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:14.117 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:54:14.117 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:54:14.118 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:54:14.174 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:54:14.176 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:54:14.179 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:54:14.180 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:54:14.180 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:54:14.180 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:54:14.181 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 22 = 2021-03-09T02:54:14.181 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.8 2021-03-09T02:54:14.981 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:54:14.982 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:14.982 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:14.982 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:14.982 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:54:14.982 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:54:14.983 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:54:15.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:54:15.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:54:15.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:54:15.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:54:15.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:54:15.061 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:54:15.061 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 22 = 2021-03-09T02:54:15.061 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1.6 2021-03-09T02:54:16.662 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:54:16.662 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:16.662 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:16.662 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:16.663 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:54:16.663 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:54:16.663 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:54:16.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:54:16.754 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:54:16.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:54:16.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:54:16.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:54:16.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:54:16.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 22 = 2021-03-09T02:54:16.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 2 2021-03-09T02:54:18.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:54:18.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:18.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:18.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:18.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:54:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:54:18.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:54:18.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:54:18.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=22 2021-03-09T02:54:18.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 22 = 22 2021-03-09T02:54:18.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:54:18.831 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snap_present cluster1 mirror test snap5 2021-03-09T02:54:18.832 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:18.832 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:18.832 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:18.833 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap5 2021-03-09T02:54:18.833 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:54:18.833 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:54:19.831 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 info mirror/test@snap5 2021-03-09T02:54:19.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:rbd image 'test': 2021-03-09T02:54:19.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: size 512 MiB in 128 objects 2021-03-09T02:54:19.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: order 22 (4 MiB objects) 2021-03-09T02:54:19.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: snapshot_count: 10 2021-03-09T02:54:19.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: id: 10a0f27e4330 2021-03-09T02:54:19.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: block_name_prefix: rbd_data.10a0f27e4330 2021-03-09T02:54:19.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: format: 2 2021-03-09T02:54:19.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: features: layering, exclusive-lock, non-primary 2021-03-09T02:54:19.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: op_features: 2021-03-09T02:54:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: flags: 2021-03-09T02:54:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: create_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:54:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: access_timestamp: Tue Mar 9 02:52:42 2021 2021-03-09T02:54:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: modify_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:54:19.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: protected: False 2021-03-09T02:54:19.901 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring state: enabled 2021-03-09T02:54:19.901 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring mode: snapshot 2021-03-09T02:54:19.901 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring global id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:54:19.901 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring primary: false 2021-03-09T02:54:19.903 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:54:19.904 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snap4-clean ] 2021-03-09T02:54:19.904 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap4-clean 2021-03-09T02:54:19.904 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:19.904 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap4-clean 2021-03-09T02:54:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:54:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:54:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:54:19.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:54:19.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:54:20.572 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:54:20.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:54:21.489 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:54:21.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:54:21.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:54:21.896 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:54:21.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap5 2021-03-09T02:54:21.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:21.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:21.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap5 2021-03-09T02:54:21.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:54:21.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:54:21.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:54:21.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:54:21.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:54:22.557 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:54:22.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:54:23.420 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:54:23.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:54:23.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:54:23.833 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:54:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ clean_snap_name=snap5-clean 2021-03-09T02:54:23.834 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap5-clean 2021-03-09T02:54:23.835 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:54:23.835 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:23.835 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:23.835 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap5-clean 2021-03-09T02:54:23.836 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap5-clean --debug-rbd=20 --debug-journaler=20 2021-03-09T02:54:24.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror test 2021-03-09T02:54:24.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:54:24.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:24.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T02:54:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=5 2021-03-09T02:54:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:54:24.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-03-09T02:54:24.840 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 72262656 bytes 2021-03-09T02:54:25.116 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 126643200 bytes 2021-03-09T02:54:25.296 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 153523200 bytes 2021-03-09T02:54:25.491 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 184968192 bytes 2021-03-09T02:54:25.737 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 241093632 bytes 2021-03-09T02:54:25.951 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 282099712 bytes 2021-03-09T02:54:26.260 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 362382336 bytes 2021-03-09T02:54:26.440 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 408120320 bytes 2021-03-09T02:54:26.680 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 456480768 bytes 2021-03-09T02:54:26.885 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 493280256 bytes 2021-03-09T02:54:27.053 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 515346432 bytes 2021-03-09T02:54:27.361 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 594143232 bytes 2021-03-09T02:54:27.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 659082240 bytes 2021-03-09T02:54:27.863 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 708481024 bytes 2021-03-09T02:54:28.083 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 751293440 bytes 2021-03-09T02:54:28.223 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 763602944 bytes 2021-03-09T02:54:28.508 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 836772864 bytes 2021-03-09T02:54:28.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 869036032 bytes 2021-03-09T02:54:28.895 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 912734208 bytes 2021-03-09T02:54:29.145 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 988118016 bytes 2021-03-09T02:54:29.361 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 1018759168 bytes 2021-03-09T02:54:29.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T02:54:29.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:54:29.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T02:54:29.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:54:29.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:54:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:29.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:54:29.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:54:29.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:54:29.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:54:30.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:54:30.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:30.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:30.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:54:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:54:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:54:30.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-03-09T02:54:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.mirror_status 2021-03-09T02:54:30.406 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:54:30.408 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:54:30.408 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-03-09T02:54:30.408 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ tee /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:54:30.476 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:test: 2021-03-09T02:54:30.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: global_id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:54:30.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+replaying 2021-03-09T02:54:30.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: replaying, {"bytes_per_second":16481280.0,"bytes_per_snapshot":481085440.0,"local_snapshot_timestamp":1615258453,"remote_snapshot_timestamp":1615258453,"replay_state":"idle"} 2021-03-09T02:54:30.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: service: mirror.1 on smithi144 2021-03-09T02:54:30.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:54:27 2021-03-09T02:54:30.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: peer_sites: 2021-03-09T02:54:30.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: name: cluster2 2021-03-09T02:54:30.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+stopped 2021-03-09T02:54:30.479 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: local image is primary 2021-03-09T02:54:30.479 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:54:28 2021-03-09T02:54:30.486 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:54:30.487 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: state: up+replaying 2021-03-09T02:54:30.488 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ description: .* /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:54:30.489 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: description: replaying, {"bytes_per_second":16481280.0,"bytes_per_snapshot":481085440.0,"local_snapshot_timestamp":1615258453,"remote_snapshot_timestamp":1615258453,"replay_state":"idle"} 2021-03-09T02:54:30.490 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:54:30.490 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:54:30.490 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^up+ 2021-03-09T02:54:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:54:30.491 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep service: *mirror..* on /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:54:30.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: service: mirror.1 on smithi144 2021-03-09T02:54:30.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:54:30.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update=2021-03-09 02:54:27 2021-03-09T02:54:30.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-03-09 02:54:27 2021-03-09T02:54:30.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:54:30.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:30.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:30.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:54:30.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local prev_last_update=2021-03-09 02:54:27 2021-03-09T02:54:30.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror.mirror_status 2021-03-09T02:54:30.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror.mirror_status 2021-03-09T02:54:30.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:54:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror.mirror_status 2021-03-09T02:54:30.506 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-03-09T02:54:30.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd: failed to query services: (13) Permission denied 2021-03-09T02:54:30.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update state 2021-03-09T02:54:30.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-03-09T02:54:30.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ last_update=2021-03-09 02:54:27 2021-03-09T02:54:30.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-03-09T02:54:30.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ state=up+replaying 2021-03-09T02:54:30.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:54:30.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep up+replaying 2021-03-09T02:54:30.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:54:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:54:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:54:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap6 2021-03-09T02:54:30.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap6 2021-03-09T02:54:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:54:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:54:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:54:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap6 2021-03-09T02:54:30.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6 --debug-rbd=20 --debug-journaler=20 2021-03-09T02:55:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:55:00.413 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:00.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:00.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:00.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:55:00.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:00.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:00.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:00.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:55:00.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:55:00.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:55:01.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:55:01.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:01.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:55:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:55:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:55:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:55:01.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:55:01.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + shift 2021-03-09T02:55:01.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:01.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance=cluster1 2021-03-09T02:55:01.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ greplocal -i loop_instance 2021-03-09T02:55:01.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state 2021-03-09T02:55:01.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0 2021-03-09T02:55:01.421 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:55:01.421 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:55:01.421 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:55:01.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:55:01.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:55:01.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:55:01.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:55:01.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:55:01.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:55:01.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:55:01.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:01.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:55:01.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:55:01.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:55:01.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:01.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:55:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:55:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:55:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:55:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:01.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:55:01.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:55:01.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:55:01.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:55:01.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:55:01.427 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:01.427 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:55:01.427 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:55:01.427 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:55:01.428 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:55:01.428 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:55:01.428 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:55:01.429 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:55:01.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:55:01.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:55:01.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:55:01.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:55:01.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:55:01.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:55:01.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:55:01.547 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:55:01.547 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:55:01.547 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:55:01.547 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:55:01.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:55:01.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:55:01.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:55:01.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:55:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:55:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:55:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:55:01.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:01.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:55:01.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:55:01.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:55:01.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:55:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:55:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:55:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:55:01.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:55:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:55:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:55:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:55:01.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:55:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:55:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:55:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:55:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:55:01.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:55:01.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:55:01.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:55:01.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:55:01.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:55:01.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:55:01.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo "state": "Replaying" 2021-03-09T02:55:01.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep -i Replaying 2021-03-09T02:55:01.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:55:01.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:55:01.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:55:01.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:55:01.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-03-09T02:55:01.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:55:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:55:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:01.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = journal ] 2021-03-09T02:55:01.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = snapshot ] 2021-03-09T02:55:01.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-03-09T02:55:01.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:55:01.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:55:01.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:01.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:01.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster2-mirror-test.status 2021-03-09T02:55:01.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster2-mirror-test.status 2021-03-09T02:55:01.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:55:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:55:01.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.status 2021-03-09T02:55:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.status 2021-03-09T02:55:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:55:01.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:01.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-03-09T02:55:01.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:55:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:01.701 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:01.702 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-03-09T02:55:02.427 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Snapshot ID: 26 2021-03-09T02:55:02.445 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:55:02.445 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:55:02.446 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:02.446 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:02.446 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:55:02.446 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-03-09T02:55:02.447 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:55:02.503 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-03-09T02:55:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snapshot_id=26 2021-03-09T02:55:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:55:02.507 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.2 2021-03-09T02:55:02.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:02.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:02.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:02.711 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:55:02.712 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:55:02.780 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:55:02.786 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=22 2021-03-09T02:55:02.786 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 26 = 22 2021-03-09T02:55:02.786 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.4 2021-03-09T02:55:03.188 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:03.188 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:03.188 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:03.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:03.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:03.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:55:03.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:55:03.253 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:55:03.253 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:55:03.256 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:55:03.256 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:55:03.257 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:55:03.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:55:03.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 26 = 2021-03-09T02:55:03.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.8 2021-03-09T02:55:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:55:04.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:55:04.131 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:55:04.132 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:55:04.136 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:55:04.136 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:55:04.137 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:55:04.138 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:55:04.138 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 26 = 2021-03-09T02:55:04.138 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1.6 2021-03-09T02:55:05.739 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:05.739 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:05.739 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:05.739 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:05.740 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:05.740 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:55:05.740 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:55:05.908 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:55:05.908 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:55:05.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:55:05.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:55:05.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:55:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:55:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 26 = 2021-03-09T02:55:05.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 2 2021-03-09T02:55:07.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:07.913 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:07.913 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:07.914 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:07.915 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:07.915 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:55:07.915 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:55:07.982 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:55:07.987 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=26 2021-03-09T02:55:07.987 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 26 = 26 2021-03-09T02:55:07.987 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:55:07.988 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snap_present cluster1 mirror test snap6 2021-03-09T02:55:07.988 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:07.988 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:07.988 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:07.988 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap6 2021-03-09T02:55:07.989 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:55:07.989 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:55:08.988 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 info mirror/test@snap6 2021-03-09T02:55:09.051 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:rbd image 'test': 2021-03-09T02:55:09.051 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: size 512 MiB in 128 objects 2021-03-09T02:55:09.051 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: order 22 (4 MiB objects) 2021-03-09T02:55:09.052 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: snapshot_count: 12 2021-03-09T02:55:09.052 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: id: 10a0f27e4330 2021-03-09T02:55:09.052 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: block_name_prefix: rbd_data.10a0f27e4330 2021-03-09T02:55:09.052 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: format: 2 2021-03-09T02:55:09.053 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: features: layering, exclusive-lock, non-primary 2021-03-09T02:55:09.053 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: op_features: 2021-03-09T02:55:09.053 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: flags: 2021-03-09T02:55:09.053 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: create_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:55:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: access_timestamp: Tue Mar 9 02:54:20 2021 2021-03-09T02:55:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: modify_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:55:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: protected: False 2021-03-09T02:55:09.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring state: enabled 2021-03-09T02:55:09.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring mode: snapshot 2021-03-09T02:55:09.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring global id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:55:09.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring primary: false 2021-03-09T02:55:09.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:55:09.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snap5-clean ] 2021-03-09T02:55:09.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap5-clean 2021-03-09T02:55:09.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:09.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:09.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap5-clean 2021-03-09T02:55:09.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:55:09.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:55:09.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:55:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:55:09.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:55:09.722 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:55:09.725 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:55:10.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:55:10.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:55:10.886 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:55:11.015 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:55:11.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap6 2021-03-09T02:55:11.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:11.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:11.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap6 2021-03-09T02:55:11.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:55:11.018 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:55:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:55:11.019 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:55:11.021 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:55:11.674 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:55:11.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:55:12.557 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:55:12.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:55:12.841 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:55:12.968 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:55:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ clean_snap_name=snap6-clean 2021-03-09T02:55:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap6-clean 2021-03-09T02:55:12.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:55:12.970 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:12.970 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:12.971 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap6-clean 2021-03-09T02:55:12.971 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap6-clean --debug-rbd=20 --debug-journaler=20 2021-03-09T02:55:13.126 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7542663440215214) > weight (0.25) 2021-03-09T02:55:13.127 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9989245908398886) > weight (0.25) 2021-03-09T02:55:13.127 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7495486166578202) > weight (0.25) 2021-03-09T02:55:13.127 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.36392526894499977) > weight (0.25) 2021-03-09T02:55:13.127 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-03-09T02:55:13.291 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror test 2021-03-09T02:55:13.291 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:55:13.292 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:13.292 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:13.292 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T02:55:13.293 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=11 2021-03-09T02:55:13.293 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:55:13.293 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-03-09T02:55:13.809 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 90683392 bytes 2021-03-09T02:55:14.112 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 148670464 bytes 2021-03-09T02:55:14.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 195257344 bytes 2021-03-09T02:55:14.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 270565376 bytes 2021-03-09T02:55:14.835 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 315338752 bytes 2021-03-09T02:55:15.112 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 367477760 bytes 2021-03-09T02:55:15.362 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 422049792 bytes 2021-03-09T02:55:15.817 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 492860416 bytes 2021-03-09T02:55:15.928 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 493473792 bytes 2021-03-09T02:55:16.063 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 508331008 bytes 2021-03-09T02:55:16.224 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 533554176 bytes 2021-03-09T02:55:16.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 555572224 bytes 2021-03-09T02:55:16.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 594390016 bytes 2021-03-09T02:55:16.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 673168384 bytes 2021-03-09T02:55:17.075 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 696263680 bytes 2021-03-09T02:55:17.201 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 700468224 bytes 2021-03-09T02:55:17.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 754842624 bytes 2021-03-09T02:55:17.617 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 792817664 bytes 2021-03-09T02:55:17.977 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 893537280 bytes 2021-03-09T02:55:18.196 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 923322368 bytes 2021-03-09T02:55:18.317 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 924182528 bytes 2021-03-09T02:55:18.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 948999168 bytes 2021-03-09T02:55:18.719 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 998897664 bytes 2021-03-09T02:55:18.917 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1029181440 bytes 2021-03-09T02:55:19.061 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1039419392 bytes 2021-03-09T02:55:19.231 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1061133312 bytes 2021-03-09T02:55:19.434 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1096389632 bytes 2021-03-09T02:55:19.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1111665664 bytes 2021-03-09T02:55:19.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1166077952 bytes 2021-03-09T02:55:20.029 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1189025792 bytes 2021-03-09T02:55:20.299 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1259258880 bytes 2021-03-09T02:55:20.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1320505344 bytes 2021-03-09T02:55:20.703 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1340574720 bytes 2021-03-09T02:55:20.942 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1396796416 bytes 2021-03-09T02:55:21.098 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1413991424 bytes 2021-03-09T02:55:21.287 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1451819008 bytes 2021-03-09T02:55:21.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1507582976 bytes 2021-03-09T02:55:21.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1543506944 bytes 2021-03-09T02:55:21.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1549229056 bytes 2021-03-09T02:55:22.015 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4000 IOs, 1549861888 bytes 2021-03-09T02:55:22.211 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4100 IOs, 1600582656 bytes 2021-03-09T02:55:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4200 IOs, 1601199104 bytes 2021-03-09T02:55:22.788 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4300 IOs, 1723627520 bytes 2021-03-09T02:55:22.962 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4400 IOs, 1741015040 bytes 2021-03-09T02:55:23.184 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4500 IOs, 1777964032 bytes 2021-03-09T02:55:23.407 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4600 IOs, 1814210560 bytes 2021-03-09T02:55:23.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4700 IOs, 1837343744 bytes 2021-03-09T02:55:23.764 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4800 IOs, 1877286912 bytes 2021-03-09T02:55:23.989 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4900 IOs, 1900603392 bytes 2021-03-09T02:55:24.188 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5000 IOs, 1935429632 bytes 2021-03-09T02:55:24.297 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T02:55:24.298 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:55:24.298 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T02:55:24.298 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:55:24.298 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:55:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:55:24.299 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:55:24.300 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:55:24.300 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:55:25.299 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:55:25.299 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:25.299 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:25.300 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:25.300 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:55:25.300 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:55:25.300 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:55:25.301 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-03-09T02:55:25.301 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.mirror_status 2021-03-09T02:55:25.301 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:55:25.302 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:55:25.302 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-03-09T02:55:25.302 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ tee /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:55:25.376 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:test: 2021-03-09T02:55:25.376 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: global_id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:55:25.377 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+replaying 2021-03-09T02:55:25.377 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":481085440.0,"local_snapshot_timestamp":1615258453,"remote_snapshot_timestamp":1615258453,"replay_state":"idle"} 2021-03-09T02:55:25.377 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: service: mirror.1 on smithi144 2021-03-09T02:55:25.377 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:54:57 2021-03-09T02:55:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: peer_sites: 2021-03-09T02:55:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: name: cluster2 2021-03-09T02:55:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+stopped 2021-03-09T02:55:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: local image is primary 2021-03-09T02:55:25.378 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:54:58 2021-03-09T02:55:25.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:55:25.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: state: up+replaying 2021-03-09T02:55:25.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ description: .* /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:55:25.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":481085440.0,"local_snapshot_timestamp":1615258453,"remote_snapshot_timestamp":1615258453,"replay_state":"idle"} 2021-03-09T02:55:25.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:55:25.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:55:25.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^up+ 2021-03-09T02:55:25.391 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:55:25.391 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep service: *mirror..* on /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:55:25.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: service: mirror.1 on smithi144 2021-03-09T02:55:25.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:55:25.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update=2021-03-09 02:54:57 2021-03-09T02:55:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-03-09 02:54:57 2021-03-09T02:55:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:25.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:25.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:55:25.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local prev_last_update=2021-03-09 02:54:57 2021-03-09T02:55:25.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror.mirror_status 2021-03-09T02:55:25.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror.mirror_status 2021-03-09T02:55:25.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:55:25.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror.mirror_status 2021-03-09T02:55:25.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-03-09T02:55:25.442 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd: failed to query services: (13) Permission denied 2021-03-09T02:55:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update state 2021-03-09T02:55:25.487 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-03-09T02:55:25.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ last_update=2021-03-09 02:54:57 2021-03-09T02:55:25.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-03-09T02:55:25.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ state=up+replaying 2021-03-09T02:55:25.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:55:25.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep up+replaying 2021-03-09T02:55:25.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:55:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:55:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:55:25.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap7 2021-03-09T02:55:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap7 2021-03-09T02:55:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:55:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:25.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:25.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap7 2021-03-09T02:55:25.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7 --debug-rbd=20 --debug-journaler=20 2021-03-09T02:55:49.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9956980995438112) > weight (0.25) 2021-03-09T02:55:49.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6813181926873185) > weight (0.25) 2021-03-09T02:55:49.020 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T02:55:49.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T02:55:49.021 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T02:55:49.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T02:55:49.019+0000 7fcfd3fff700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 22335) UID: 1000 2021-03-09T02:55:49.222 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T02:55:50.022 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T02:55:50.022 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T02:55:50.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:55:50.023 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T02:55:50.023 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T02:55:50.024 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T02:55:50.026 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T02:55:50.027 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T02:55:55.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:55:55.322 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:55.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:55.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:55.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:55:55.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:55.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:55.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:55.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:55:55.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:55:55.325 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:55:56.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:55:56.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:56.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:56.324 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:56.325 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:55:56.325 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:55:56.325 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:55:56.325 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:55:56.326 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:55:56.326 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + shift 2021-03-09T02:55:56.326 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:56.326 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance=cluster1 2021-03-09T02:55:56.326 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:grep+ -ilocal state loop_instance 2021-03-09T02:55:56.327 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0 2021-03-09T02:55:56.327 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:55:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:55:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:55:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:55:56.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:55:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:55:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:55:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:55:56.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:55:56.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:55:56.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:55:56.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:56.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:55:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:55:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:55:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:56.333 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:55:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:55:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:55:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:55:56.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:55:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:55:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:55:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:55:56.335 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:55:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:55:56.336 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:55:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:55:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:55:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:55:56.337 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:55:56.338 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:55:56.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:55:56.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:55:56.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:55:56.352 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:55:56.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:55:56.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:55:56.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:55:56.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:55:56.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:55:56.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:55:56.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:55:56.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:55:56.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:55:56.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:55:56.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:55:56.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:56.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:55:56.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:55:56.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:55:56.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:56.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:55:56.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:55:56.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:55:56.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:55:56.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:56.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:55:56.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:55:56.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:55:56.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:55:56.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:55:56.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:55:56.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:55:56.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:55:56.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:55:56.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:55:56.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:55:56.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:55:56.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:55:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:55:56.482 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:55:56.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:55:56.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:55:56.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:55:56.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:55:56.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep -i Replaying 2021-03-09T02:55:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo "state": "Replaying" 2021-03-09T02:55:56.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:55:56.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:55:56.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:55:56.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:55:56.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-03-09T02:55:56.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:55:56.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:55:56.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:56.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:56.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = journal ] 2021-03-09T02:55:56.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = snapshot ] 2021-03-09T02:55:56.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-03-09T02:55:56.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:55:56.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:55:56.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:56.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:56.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster2-mirror-test.status 2021-03-09T02:55:56.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster2-mirror-test.status 2021-03-09T02:55:56.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:55:56.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:55:56.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.status 2021-03-09T02:55:56.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.status 2021-03-09T02:55:56.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:55:56.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:56.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-03-09T02:55:56.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:55:56.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:56.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:56.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-03-09T02:55:57.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Snapshot ID: 30 2021-03-09T02:55:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:55:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:55:57.372 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:57.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:57.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:55:57.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-03-09T02:55:57.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:55:57.432 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-03-09T02:55:57.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snapshot_id=30 2021-03-09T02:55:57.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:55:57.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.2 2021-03-09T02:55:57.639 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:57.640 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:57.640 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:57.640 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:57.641 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:57.641 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:55:57.642 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:55:57.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:55:57.715 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=26 2021-03-09T02:55:57.715 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 30 = 26 2021-03-09T02:55:57.716 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.4 2021-03-09T02:55:58.117 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:58.117 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:58.118 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:58.118 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:58.118 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:58.118 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:55:58.119 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:55:58.193 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:55:58.193 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:55:58.196 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:55:58.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:55:58.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:55:58.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:55:58.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 30 = 2021-03-09T02:55:58.198 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.8 2021-03-09T02:55:58.999 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:58.999 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:55:58.999 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:55:58.999 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:55:59.000 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:55:59.000 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:55:59.000 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:55:59.066 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:55:59.067 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:55:59.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:55:59.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:55:59.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:55:59.072 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:55:59.072 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 30 = 2021-03-09T02:55:59.072 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1.6 2021-03-09T02:56:00.673 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:00.674 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:56:00.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:56:00.736 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:56:00.736 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:56:00.741 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:56:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:56:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:56:00.742 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:56:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 30 = 2021-03-09T02:56:00.743 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 2 2021-03-09T02:56:02.745 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:02.745 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:02.745 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:02.746 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:02.746 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:02.746 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:56:02.746 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:56:02.821 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:56:02.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=30 2021-03-09T02:56:02.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 30 = 30 2021-03-09T02:56:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:56:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snap_present cluster1 mirror test snap7 2021-03-09T02:56:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:02.828 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:02.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap7 2021-03-09T02:56:02.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:56:02.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:56:03.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 info mirror/test@snap7 2021-03-09T02:56:03.894 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:rbd image 'test': 2021-03-09T02:56:03.895 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: size 512 MiB in 128 objects 2021-03-09T02:56:03.895 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: order 22 (4 MiB objects) 2021-03-09T02:56:03.895 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: snapshot_count: 14 2021-03-09T02:56:03.895 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: id: 10a0f27e4330 2021-03-09T02:56:03.895 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: block_name_prefix: rbd_data.10a0f27e4330 2021-03-09T02:56:03.896 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: format: 2 2021-03-09T02:56:03.896 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: features: layering, exclusive-lock, non-primary 2021-03-09T02:56:03.896 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: op_features: 2021-03-09T02:56:03.896 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: flags: 2021-03-09T02:56:03.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: create_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:56:03.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: access_timestamp: Tue Mar 9 02:54:20 2021 2021-03-09T02:56:03.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: modify_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:56:03.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: protected: False 2021-03-09T02:56:03.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring state: enabled 2021-03-09T02:56:03.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring mode: snapshot 2021-03-09T02:56:03.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring global id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:56:03.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring primary: false 2021-03-09T02:56:03.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:56:03.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snap6-clean ] 2021-03-09T02:56:03.901 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap6-clean 2021-03-09T02:56:03.901 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:03.901 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:03.901 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap6-clean 2021-03-09T02:56:03.901 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:56:03.902 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:56:03.902 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:56:03.902 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:56:03.903 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:56:04.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:56:04.572 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:56:05.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:56:05.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:56:05.798 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:56:05.926 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:56:05.926 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap7 2021-03-09T02:56:05.927 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:05.927 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:05.927 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap7 2021-03-09T02:56:05.927 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:56:05.928 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:56:05.928 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:56:05.928 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:56:05.929 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:56:06.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:56:06.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:56:07.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:56:07.508 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:56:07.784 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:56:07.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:56:07.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ clean_snap_name=snap7-clean 2021-03-09T02:56:07.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap7-clean 2021-03-09T02:56:07.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:56:07.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:07.913 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:07.913 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap7-clean 2021-03-09T02:56:07.913 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap7-clean --debug-rbd=20 --debug-journaler=20 2021-03-09T02:56:08.379 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror test 2021-03-09T02:56:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:56:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:08.380 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T02:56:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=13 2021-03-09T02:56:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:56:08.381 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-03-09T02:56:08.799 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 47901696 bytes 2021-03-09T02:56:09.024 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 72328192 bytes 2021-03-09T02:56:09.207 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 99214336 bytes 2021-03-09T02:56:09.428 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 132125696 bytes 2021-03-09T02:56:09.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 167268352 bytes 2021-03-09T02:56:09.872 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 211396608 bytes 2021-03-09T02:56:10.134 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 274705408 bytes 2021-03-09T02:56:10.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 365198336 bytes 2021-03-09T02:56:10.849 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 458609664 bytes 2021-03-09T02:56:11.011 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 473478144 bytes 2021-03-09T02:56:11.137 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 474090496 bytes 2021-03-09T02:56:11.360 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 525532160 bytes 2021-03-09T02:56:11.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 544147456 bytes 2021-03-09T02:56:11.839 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 616399872 bytes 2021-03-09T02:56:11.975 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 621759488 bytes 2021-03-09T02:56:12.126 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 635843584 bytes 2021-03-09T02:56:12.339 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 672202752 bytes 2021-03-09T02:56:12.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 731535360 bytes 2021-03-09T02:56:12.763 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 744241152 bytes 2021-03-09T02:56:12.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 756172800 bytes 2021-03-09T02:56:13.127 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 812272640 bytes 2021-03-09T02:56:13.362 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 841731072 bytes 2021-03-09T02:56:13.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 880925696 bytes 2021-03-09T02:56:13.793 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 919266304 bytes 2021-03-09T02:56:13.970 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 942432256 bytes 2021-03-09T02:56:14.169 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 970621952 bytes 2021-03-09T02:56:14.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 991528960 bytes 2021-03-09T02:56:14.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1050189824 bytes 2021-03-09T02:56:14.820 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1082852352 bytes 2021-03-09T02:56:15.011 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1107766272 bytes 2021-03-09T02:56:15.372 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1210933248 bytes 2021-03-09T02:56:15.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1296000000 bytes 2021-03-09T02:56:15.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1332002816 bytes 2021-03-09T02:56:16.173 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1383419904 bytes 2021-03-09T02:56:16.330 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1394589696 bytes 2021-03-09T02:56:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1448838144 bytes 2021-03-09T02:56:16.853 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1498114048 bytes 2021-03-09T02:56:17.148 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1569576960 bytes 2021-03-09T02:56:17.285 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1573211136 bytes 2021-03-09T02:56:17.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4000 IOs, 1632843776 bytes 2021-03-09T02:56:17.793 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4100 IOs, 1672852480 bytes 2021-03-09T02:56:18.082 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4200 IOs, 1736102912 bytes 2021-03-09T02:56:18.264 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4300 IOs, 1755750400 bytes 2021-03-09T02:56:18.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4400 IOs, 1817676800 bytes 2021-03-09T02:56:18.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4500 IOs, 1879982080 bytes 2021-03-09T02:56:19.039 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4600 IOs, 1914577920 bytes 2021-03-09T02:56:19.337 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4700 IOs, 1985475584 bytes 2021-03-09T02:56:19.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4800 IOs, 2013099008 bytes 2021-03-09T02:56:19.731 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4900 IOs, 2034932736 bytes 2021-03-09T02:56:20.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5000 IOs, 2101353472 bytes 2021-03-09T02:56:20.377 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5100 IOs, 2197048320 bytes 2021-03-09T02:56:20.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5200 IOs, 2230840320 bytes 2021-03-09T02:56:20.762 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5300 IOs, 2242070528 bytes 2021-03-09T02:56:21.036 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5400 IOs, 2303086592 bytes 2021-03-09T02:56:21.337 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5500 IOs, 2370315264 bytes 2021-03-09T02:56:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T02:56:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:56:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T02:56:21.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:56:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:56:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:21.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:56:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:56:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:56:21.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:56:22.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:56:22.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:22.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:22.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:22.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:56:22.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:56:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:56:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-03-09T02:56:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.mirror_status 2021-03-09T02:56:22.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:56:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:56:22.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + CEPH_ARGS= rbd --clustertee cluster1 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:56:22.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: mirror image status mirror/test 2021-03-09T02:56:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:test: 2021-03-09T02:56:22.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: global_id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:56:22.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+replaying 2021-03-09T02:56:22.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":484429312.0,"local_snapshot_timestamp":1615258502,"remote_snapshot_timestamp":1615258502,"replay_state":"idle"} 2021-03-09T02:56:22.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: service: mirror.1 on smithi144 2021-03-09T02:56:22.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:55:57 2021-03-09T02:56:22.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: peer_sites: 2021-03-09T02:56:22.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: name: cluster2 2021-03-09T02:56:22.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+stopped 2021-03-09T02:56:22.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: local image is primary 2021-03-09T02:56:22.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:55:58 2021-03-09T02:56:22.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:56:22.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: state: up+replaying 2021-03-09T02:56:22.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ description: .* /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:56:22.471 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":484429312.0,"local_snapshot_timestamp":1615258502,"remote_snapshot_timestamp":1615258502,"replay_state":"idle"} 2021-03-09T02:56:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:56:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:56:22.472 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^up+ 2021-03-09T02:56:22.473 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:56:22.474 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep service: *mirror..* on /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:56:22.474 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: service: mirror.1 on smithi144 2021-03-09T02:56:22.476 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:56:22.480 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update=2021-03-09 02:55:57 2021-03-09T02:56:22.481 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-03-09 02:55:57 2021-03-09T02:56:22.481 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:22.481 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:22.481 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:56:22.482 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local prev_last_update=2021-03-09 02:55:57 2021-03-09T02:56:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror.mirror_status 2021-03-09T02:56:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror.mirror_status 2021-03-09T02:56:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:56:22.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror.mirror_status 2021-03-09T02:56:22.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-03-09T02:56:22.524 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd: failed to query services: (13) Permission denied 2021-03-09T02:56:22.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update state 2021-03-09T02:56:22.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-03-09T02:56:22.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ last_update=2021-03-09 02:55:57 2021-03-09T02:56:22.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-03-09T02:56:22.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ state=up+replaying 2021-03-09T02:56:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:56:22.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep up+replaying 2021-03-09T02:56:22.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:56:22.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:56:22.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:56:22.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap8 2021-03-09T02:56:22.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap8 2021-03-09T02:56:22.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:56:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:22.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap8 2021-03-09T02:56:22.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8 --debug-rbd=20 --debug-journaler=20 2021-03-09T02:56:48.129 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.685094409977457) > weight (0.25) 2021-03-09T02:56:48.130 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9293858146555923) > weight (0.25) 2021-03-09T02:56:48.130 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T02:56:48.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T02:56:48.131 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T02:56:48.132 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T02:56:48.129+0000 7f5f0f387700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 23822) UID: 1000 2021-03-09T02:56:48.332 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T02:56:51.391 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:56:51.391 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:51.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:51.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:51.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:56:51.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:51.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:51.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:51.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:56:51.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:56:51.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:56:52.392 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:56:52.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:52.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:52.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:52.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:56:52.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:56:52.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:56:52.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:56:52.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:56:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:56:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + local instance=cluster1 2021-03-09T02:56:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loop_instance 2021-03-09T02:56:52.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0 2021-03-09T02:56:52.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:grep -i state 2021-03-09T02:56:52.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:56:52.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:56:52.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:56:52.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:56:52.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:56:52.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:56:52.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:56:52.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:56:52.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:56:52.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:56:52.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:56:52.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:56:52.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:56:52.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:56:52.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:56:52.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:56:52.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:56:52.403 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:56:52.403 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:56:52.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:52.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:56:52.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:56:52.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:56:52.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:56:52.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:56:52.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:56:52.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:56:52.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:56:52.406 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:56:52.406 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:56:52.406 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:56:52.406 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:56:52.407 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:56:52.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:56:52.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:56:52.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:56:52.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:56:52.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:56:52.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:56:52.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:56:52.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:56:52.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:56:52.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:56:52.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:56:52.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:56:52.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:56:52.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:56:52.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:56:52.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:56:52.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:56:52.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:56:52.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:56:52.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:56:52.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:56:52.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:56:52.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:56:52.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:56:52.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:52.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:56:52.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:56:52.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:56:52.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:56:52.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:56:52.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:56:52.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:56:52.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:56:52.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:56:52.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:56:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:56:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:56:52.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:56:52.547 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:56:52.547 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:56:52.547 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:56:52.547 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:56:52.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:56:52.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:56:52.658 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo "state": "Replaying" 2021-03-09T02:56:52.658 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep -i Replaying 2021-03-09T02:56:52.659 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:56:52.661 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:56:52.661 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:56:52.661 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:56:52.661 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-03-09T02:56:52.662 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:56:52.662 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:56:52.662 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:52.662 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:52.663 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = journal ] 2021-03-09T02:56:52.663 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = snapshot ] 2021-03-09T02:56:52.663 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-03-09T02:56:52.663 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:56:52.664 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:56:52.664 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:52.664 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:52.664 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster2-mirror-test.status 2021-03-09T02:56:52.664 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster2-mirror-test.status 2021-03-09T02:56:52.665 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:56:52.666 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:56:52.666 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.status 2021-03-09T02:56:52.667 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.status 2021-03-09T02:56:52.667 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:56:52.667 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:52.667 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-03-09T02:56:52.668 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:56:52.668 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:52.668 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:52.668 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-03-09T02:56:53.131 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T02:56:53.132 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T02:56:53.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:56:53.133 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T02:56:53.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T02:56:53.133 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T02:56:53.136 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T02:56:53.136 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T02:56:53.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Snapshot ID: 34 2021-03-09T02:56:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:56:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:56:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:53.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:53.444 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:56:53.444 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-03-09T02:56:53.444 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:56:53.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-03-09T02:56:53.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snapshot_id=34 2021-03-09T02:56:53.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:56:53.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.2 2021-03-09T02:56:53.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:53.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:53.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:53.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:53.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:53.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:56:53.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:56:53.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:56:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=30 2021-03-09T02:56:53.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 34 = 30 2021-03-09T02:56:53.784 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.4 2021-03-09T02:56:54.185 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:54.186 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:54.186 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:54.186 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:54.186 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:54.187 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:56:54.187 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:56:54.249 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:56:54.249 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:56:54.252 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:56:54.253 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:56:54.253 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:56:54.255 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:56:54.255 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 34 = 2021-03-09T02:56:54.256 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.8 2021-03-09T02:56:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:55.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:55.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:56:55.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:56:55.131 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:56:55.132 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:56:55.136 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:56:55.136 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:56:55.137 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:56:55.137 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:56:55.137 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 34 = 2021-03-09T02:56:55.138 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1.6 2021-03-09T02:56:56.739 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:56.739 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:56.739 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:56.740 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:56.740 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:56.740 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:56:56.740 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:56:56.967 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:56:56.968 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:56:56.971 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:56:56.971 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:56:56.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:56:56.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:56:56.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 34 = 2021-03-09T02:56:56.973 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 2 2021-03-09T02:56:58.974 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:58.974 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:58.974 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:58.975 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:58.975 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:56:58.975 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:56:58.975 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:56:59.051 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:56:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=34 2021-03-09T02:56:59.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 34 = 34 2021-03-09T02:56:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:56:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snap_present cluster1 mirror test snap8 2021-03-09T02:56:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:56:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:56:59.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:56:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap8 2021-03-09T02:56:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:56:59.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:57:00.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 info mirror/test@snap8 2021-03-09T02:57:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:rbd image 'test': 2021-03-09T02:57:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: size 512 MiB in 128 objects 2021-03-09T02:57:00.119 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: order 22 (4 MiB objects) 2021-03-09T02:57:00.120 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: snapshot_count: 16 2021-03-09T02:57:00.120 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: id: 10a0f27e4330 2021-03-09T02:57:00.120 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: block_name_prefix: rbd_data.10a0f27e4330 2021-03-09T02:57:00.120 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: format: 2 2021-03-09T02:57:00.121 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: features: layering, exclusive-lock, non-primary 2021-03-09T02:57:00.121 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: op_features: 2021-03-09T02:57:00.121 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: flags: 2021-03-09T02:57:00.121 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: create_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:57:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: access_timestamp: Tue Mar 9 02:56:04 2021 2021-03-09T02:57:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: modify_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:57:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: protected: False 2021-03-09T02:57:00.122 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring state: enabled 2021-03-09T02:57:00.123 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring mode: snapshot 2021-03-09T02:57:00.123 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring global id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:57:00.123 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring primary: false 2021-03-09T02:57:00.125 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:57:00.126 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snap7-clean ] 2021-03-09T02:57:00.126 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap7-clean 2021-03-09T02:57:00.126 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:00.127 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:00.127 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap7-clean 2021-03-09T02:57:00.127 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:57:00.127 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:57:00.128 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:00.128 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:00.128 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:57:00.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:57:00.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:01.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:57:01.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:01.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:57:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap8 2021-03-09T02:57:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:02.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:02.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap8 2021-03-09T02:57:02.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:57:02.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:57:02.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:02.018 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:02.018 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:57:02.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:57:02.679 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:03.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:57:03.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:03.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:03.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:57:03.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ clean_snap_name=snap8-clean 2021-03-09T02:57:03.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap8-clean 2021-03-09T02:57:03.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:57:03.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:03.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:03.898 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap8-clean 2021-03-09T02:57:03.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap8-clean --debug-rbd=20 --debug-journaler=20 2021-03-09T02:57:04.027 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9187093561900542) > weight (0.25) 2021-03-09T02:57:04.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.27663572830418004) > weight (0.25) 2021-03-09T02:57:04.028 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T02:57:04.029 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T02:57:04.030 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T02:57:04.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T02:57:04.025+0000 7fc24d838700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 25356) UID: 1000 2021-03-09T02:57:04.231 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T02:57:04.452 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror test 2021-03-09T02:57:04.452 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:57:04.452 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:04.452 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:04.453 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T02:57:04.453 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=14 2021-03-09T02:57:04.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:57:04.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-03-09T02:57:04.958 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 79142912 bytes 2021-03-09T02:57:05.284 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 145882112 bytes 2021-03-09T02:57:05.490 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 178793472 bytes 2021-03-09T02:57:05.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 179610624 bytes 2021-03-09T02:57:05.796 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 184517632 bytes 2021-03-09T02:57:06.061 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 249757696 bytes 2021-03-09T02:57:06.264 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 276805632 bytes 2021-03-09T02:57:06.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 294477824 bytes 2021-03-09T02:57:06.726 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 356178944 bytes 2021-03-09T02:57:06.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 391217152 bytes 2021-03-09T02:57:07.145 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 422042624 bytes 2021-03-09T02:57:07.451 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 503498752 bytes 2021-03-09T02:57:07.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 546372608 bytes 2021-03-09T02:57:07.834 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 547007488 bytes 2021-03-09T02:57:08.031 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 585569280 bytes 2021-03-09T02:57:08.284 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 631087104 bytes 2021-03-09T02:57:08.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 662837248 bytes 2021-03-09T02:57:08.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 713304064 bytes 2021-03-09T02:57:08.945 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 733325312 bytes 2021-03-09T02:57:09.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 794268672 bytes 2021-03-09T02:57:09.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 858265600 bytes 2021-03-09T02:57:09.723 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 907461632 bytes 2021-03-09T02:57:10.033 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 969535488 bytes 2021-03-09T02:57:10.298 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1008903168 bytes 2021-03-09T02:57:10.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1041861632 bytes 2021-03-09T02:57:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1102093312 bytes 2021-03-09T02:57:10.974 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1121815552 bytes 2021-03-09T02:57:11.229 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1169384448 bytes 2021-03-09T02:57:11.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1235657728 bytes 2021-03-09T02:57:11.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1251920896 bytes 2021-03-09T02:57:11.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1288233984 bytes 2021-03-09T02:57:12.447 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1345006592 bytes 2021-03-09T02:57:12.672 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1383506944 bytes 2021-03-09T02:57:12.848 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1400342528 bytes 2021-03-09T02:57:13.012 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1411110912 bytes 2021-03-09T02:57:13.031 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T02:57:13.032 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T02:57:13.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:57:13.032 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T02:57:13.033 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T02:57:13.033 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T02:57:13.035 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T02:57:13.035 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T02:57:13.274 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1461493760 bytes 2021-03-09T02:57:13.521 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1484007424 bytes 2021-03-09T02:57:13.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1548793856 bytes 2021-03-09T02:57:14.107 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1603848192 bytes 2021-03-09T02:57:14.430 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4000 IOs, 1683987456 bytes 2021-03-09T02:57:14.811 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4100 IOs, 1782814720 bytes 2021-03-09T02:57:14.996 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4200 IOs, 1797042176 bytes 2021-03-09T02:57:15.365 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4300 IOs, 1894537216 bytes 2021-03-09T02:57:15.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4400 IOs, 1947254784 bytes 2021-03-09T02:57:15.927 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4500 IOs, 2020642816 bytes 2021-03-09T02:57:16.295 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4600 IOs, 2119944192 bytes 2021-03-09T02:57:16.490 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4700 IOs, 2140406784 bytes 2021-03-09T02:57:16.676 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4800 IOs, 2156813312 bytes 2021-03-09T02:57:16.859 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4900 IOs, 2179546112 bytes 2021-03-09T02:57:17.096 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5000 IOs, 2198571008 bytes 2021-03-09T02:57:17.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5100 IOs, 2257902592 bytes 2021-03-09T02:57:17.702 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5200 IOs, 2347807744 bytes 2021-03-09T02:57:17.988 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5300 IOs, 2395441152 bytes 2021-03-09T02:57:18.221 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5400 IOs, 2429948928 bytes 2021-03-09T02:57:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T02:57:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:57:18.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T02:57:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:57:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:57:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:18.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:18.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:18.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:57:18.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:57:18.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:57:18.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:57:19.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:57:19.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:19.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:19.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:19.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:57:19.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:57:19.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:57:19.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-03-09T02:57:19.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.mirror_status 2021-03-09T02:57:19.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:57:19.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:57:19.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-03-09T02:57:19.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ tee /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:57:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:test: 2021-03-09T02:57:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: global_id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:57:19.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+replaying 2021-03-09T02:57:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":502059008.0,"local_snapshot_timestamp":1615258557,"remote_snapshot_timestamp":1615258557,"replay_state":"idle"} 2021-03-09T02:57:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: service: mirror.1 on smithi144 2021-03-09T02:57:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:56:57 2021-03-09T02:57:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: peer_sites: 2021-03-09T02:57:19.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: name: cluster2 2021-03-09T02:57:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+stopped 2021-03-09T02:57:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: local image is primary 2021-03-09T02:57:19.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:56:58 2021-03-09T02:57:19.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:57:19.554 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: state: up+replaying 2021-03-09T02:57:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ description: .* /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:57:19.556 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: description: replaying, {"bytes_per_second":0.0,"bytes_per_snapshot":502059008.0,"local_snapshot_timestamp":1615258557,"remote_snapshot_timestamp":1615258557,"replay_state":"idle"} 2021-03-09T02:57:19.557 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:57:19.558 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:57:19.558 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^up+ 2021-03-09T02:57:19.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:57:19.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep service: *mirror..* on /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:57:19.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: service: mirror.1 on smithi144 2021-03-09T02:57:19.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:57:19.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update=2021-03-09 02:56:57 2021-03-09T02:57:19.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-03-09 02:56:57 2021-03-09T02:57:19.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:19.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:19.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:19.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:57:19.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local prev_last_update=2021-03-09 02:56:57 2021-03-09T02:57:19.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror.mirror_status 2021-03-09T02:57:19.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror.mirror_status 2021-03-09T02:57:19.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:57:19.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror.mirror_status 2021-03-09T02:57:19.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-03-09T02:57:19.812 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd: failed to query services: (13) Permission denied 2021-03-09T02:57:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update state 2021-03-09T02:57:19.864 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-03-09T02:57:19.870 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ last_update=2021-03-09 02:56:57 2021-03-09T02:57:19.870 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-03-09T02:57:19.874 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ state=up+replaying 2021-03-09T02:57:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:57:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep up+replaying 2021-03-09T02:57:19.875 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:57:19.876 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:57:19.877 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:57:19.877 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap9 2021-03-09T02:57:19.877 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap9 2021-03-09T02:57:19.877 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:57:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:19.878 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap9 2021-03-09T02:57:19.879 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9 --debug-rbd=20 --debug-journaler=20 2021-03-09T02:57:49.408 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:57:49.408 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:49.409 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:49.409 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:49.409 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:57:49.409 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:49.409 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:49.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:49.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:57:49.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:57:49.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:57:50.412 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:57:50.412 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:50.412 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:50.412 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:50.413 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:57:50.413 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:57:50.413 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:57:50.413 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:57:50.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:57:50.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift+ 2021-03-09T02:57:50.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:50.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:grep -i state 2021-03-09T02:57:50.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance=cluster1 2021-03-09T02:57:50.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loop_instance 2021-03-09T02:57:50.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0 2021-03-09T02:57:50.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:57:50.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:57:50.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:57:50.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:57:50.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:57:50.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:57:50.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:57:50.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:57:50.419 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:57:50.419 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:57:50.419 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:57:50.419 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:57:50.419 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:57:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:57:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:57:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:57:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:57:50.420 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:57:50.421 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:57:50.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:50.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:57:50.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:57:50.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:57:50.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:57:50.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:57:50.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:57:50.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:57:50.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:57:50.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:57:50.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:57:50.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:57:50.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:57:50.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:57:50.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:57:50.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:57:50.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:57:50.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:57:50.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:57:50.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:57:50.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:57:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:57:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:57:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:57:50.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:57:50.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:57:50.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:57:50.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:57:50.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:57:50.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:57:50.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:57:50.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:57:50.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:57:50.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:57:50.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:57:50.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:57:50.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:57:50.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:57:50.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:50.554 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:57:50.554 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:57:50.554 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:57:50.554 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:57:50.555 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:57:50.555 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:57:50.555 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:57:50.556 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:57:50.556 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:57:50.556 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:57:50.556 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:57:50.557 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:57:50.557 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:57:50.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:57:50.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:57:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:57:50.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:57:50.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:57:50.676 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:57:50.676 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo "state": "Replaying" 2021-03-09T02:57:50.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep -i Replaying 2021-03-09T02:57:50.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:57:50.680 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:57:50.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:57:50.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:57:50.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-03-09T02:57:50.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:57:50.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:57:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = journal ] 2021-03-09T02:57:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = snapshot ] 2021-03-09T02:57:50.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-03-09T02:57:50.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:57:50.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:57:50.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:50.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:50.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster2-mirror-test.status 2021-03-09T02:57:50.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster2-mirror-test.status 2021-03-09T02:57:50.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:57:50.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:57:50.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.status 2021-03-09T02:57:50.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.status 2021-03-09T02:57:50.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:57:50.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:57:50.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-03-09T02:57:50.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:57:50.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:50.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:50.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-03-09T02:57:51.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Snapshot ID: 38 2021-03-09T02:57:51.445 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:57:51.445 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:57:51.446 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:51.446 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:51.446 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:57:51.446 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-03-09T02:57:51.446 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:57:51.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-03-09T02:57:51.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snapshot_id=38 2021-03-09T02:57:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:57:51.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.2 2021-03-09T02:57:51.703 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:57:51.704 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:51.704 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:51.704 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:51.704 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:57:51.705 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:57:51.705 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:57:51.777 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:57:51.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=34 2021-03-09T02:57:51.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 38 = 34 2021-03-09T02:57:51.784 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.4 2021-03-09T02:57:52.185 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:57:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:52.186 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:52.187 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:57:52.187 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:57:52.187 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:57:52.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:57:52.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:57:52.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:57:52.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:57:52.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:57:52.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:57:52.263 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 38 = 2021-03-09T02:57:52.263 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.8 2021-03-09T02:57:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:57:53.064 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:53.065 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:53.065 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:53.065 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:57:53.066 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:57:53.066 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:57:53.140 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:57:53.141 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:57:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:57:53.145 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:57:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:57:53.146 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:57:53.147 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 38 = 2021-03-09T02:57:53.147 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1.6 2021-03-09T02:57:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:57:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:54.748 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:57:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:57:54.749 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:57:54.825 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:57:54.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:57:54.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:57:54.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:57:54.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:57:54.831 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:57:54.831 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 38 = 2021-03-09T02:57:54.831 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 2 2021-03-09T02:57:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:57:56.831 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:57:56.832 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:57:56.833 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:57:56.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:57:56.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=38 2021-03-09T02:57:56.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 38 = 38 2021-03-09T02:57:56.905 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:57:56.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snap_present cluster1 mirror test snap9 2021-03-09T02:57:56.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:57:56.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:56.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:56.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap9 2021-03-09T02:57:56.908 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:57:56.908 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:57:57.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 info mirror/test@snap9 2021-03-09T02:57:57.971 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:rbd image 'test': 2021-03-09T02:57:57.971 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: size 512 MiB in 128 objects 2021-03-09T02:57:57.971 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: order 22 (4 MiB objects) 2021-03-09T02:57:57.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: snapshot_count: 18 2021-03-09T02:57:57.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: id: 10a0f27e4330 2021-03-09T02:57:57.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: block_name_prefix: rbd_data.10a0f27e4330 2021-03-09T02:57:57.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: format: 2 2021-03-09T02:57:57.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: features: layering, exclusive-lock, non-primary 2021-03-09T02:57:57.973 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: op_features: 2021-03-09T02:57:57.973 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: flags: 2021-03-09T02:57:57.973 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: create_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:57:57.973 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: access_timestamp: Tue Mar 9 02:56:04 2021 2021-03-09T02:57:57.974 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: modify_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:57:57.974 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: protected: False 2021-03-09T02:57:57.974 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring state: enabled 2021-03-09T02:57:57.974 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring mode: snapshot 2021-03-09T02:57:57.974 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring global id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:57:57.975 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring primary: false 2021-03-09T02:57:57.977 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:57:57.977 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snap8-clean ] 2021-03-09T02:57:57.977 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap8-clean 2021-03-09T02:57:57.978 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:57.978 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:57.978 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap8-clean 2021-03-09T02:57:57.978 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:57:57.979 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:57:57.979 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:57.979 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:57.980 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:57:58.662 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:57:58.666 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:59.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:57:59.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:59.815 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:59.942 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:57:59.945 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap9 2021-03-09T02:57:59.946 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:57:59.946 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:57:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap9 2021-03-09T02:57:59.947 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:57:59.948 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:57:59.948 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:59.948 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:57:59.949 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:58:00.616 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:58:00.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:58:01.628 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:58:01.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:58:01.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:58:02.041 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:58:02.041 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ clean_snap_name=snap9-clean 2021-03-09T02:58:02.042 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap9-clean 2021-03-09T02:58:02.042 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:58:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:02.043 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap9-clean 2021-03-09T02:58:02.044 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap9-clean --debug-rbd=20 --debug-journaler=20 2021-03-09T02:58:02.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5236398826954102) > weight (0.25) 2021-03-09T02:58:02.138 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7229659184513958) > weight (0.25) 2021-03-09T02:58:02.138 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T02:58:02.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T02:58:02.139 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T02:58:02.140 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T02:58:02.140 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T02:58:02.141 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T02:58:02.135+0000 7f8427ef2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 25929) UID: 1000 2021-03-09T02:58:02.341 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T02:58:02.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror test 2021-03-09T02:58:02.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:58:02.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:02.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:02.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T02:58:02.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=14 2021-03-09T02:58:02.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T02:58:02.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror test --debug-rbd=20 --debug-journaler=20 2021-03-09T02:58:02.881 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 40576000 bytes 2021-03-09T02:58:03.155 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 75752448 bytes 2021-03-09T02:58:03.473 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 144687104 bytes 2021-03-09T02:58:03.661 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 148984832 bytes 2021-03-09T02:58:03.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 204955648 bytes 2021-03-09T02:58:04.240 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 286713856 bytes 2021-03-09T02:58:04.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 344536064 bytes 2021-03-09T02:58:04.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 390197248 bytes 2021-03-09T02:58:05.095 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 435003392 bytes 2021-03-09T02:58:05.364 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 492259328 bytes 2021-03-09T02:58:05.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 528571392 bytes 2021-03-09T02:58:05.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 598904832 bytes 2021-03-09T02:58:06.063 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 599502848 bytes 2021-03-09T02:58:06.325 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 644074496 bytes 2021-03-09T02:58:06.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 650639360 bytes 2021-03-09T02:58:06.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 706913280 bytes 2021-03-09T02:58:07.038 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 760584192 bytes 2021-03-09T02:58:07.243 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 778593280 bytes 2021-03-09T02:58:07.428 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 792871936 bytes 2021-03-09T02:58:07.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 833603584 bytes 2021-03-09T02:58:07.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 850680832 bytes 2021-03-09T02:58:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 896182272 bytes 2021-03-09T02:58:08.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T02:58:08.242 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T02:58:08.242 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T02:58:08.243 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T02:58:08.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T02:58:08.246 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-03-09T02:58:08.360 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 920042496 bytes 2021-03-09T02:58:08.714 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1007237120 bytes 2021-03-09T02:58:08.982 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1047867392 bytes 2021-03-09T02:58:09.286 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1108795392 bytes 2021-03-09T02:58:09.666 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1223411712 bytes 2021-03-09T02:58:09.926 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1245565952 bytes 2021-03-09T02:58:10.192 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1295652864 bytes 2021-03-09T02:58:10.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1341530112 bytes 2021-03-09T02:58:10.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1379997696 bytes 2021-03-09T02:58:11.046 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1449356288 bytes 2021-03-09T02:58:11.065 INFO:tasks.ceph.cluster2.osd.0.smithi144.stderr:2021-03-09T02:58:11.059+0000 7f0e42124700 -1 reset not still connected to 0x5573f38ba8f0 2021-03-09T02:58:11.300 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1493807104 bytes 2021-03-09T02:58:11.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1494436864 bytes 2021-03-09T02:58:11.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1511645184 bytes 2021-03-09T02:58:11.850 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1531631616 bytes 2021-03-09T02:58:12.040 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1551226880 bytes 2021-03-09T02:58:12.290 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1586309120 bytes 2021-03-09T02:58:12.627 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1677373440 bytes 2021-03-09T02:58:12.838 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4000 IOs, 1680077824 bytes 2021-03-09T02:58:13.133 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4100 IOs, 1747137536 bytes 2021-03-09T02:58:13.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4200 IOs, 1778193408 bytes 2021-03-09T02:58:13.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4300 IOs, 1804803072 bytes 2021-03-09T02:58:13.781 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4400 IOs, 1807549440 bytes 2021-03-09T02:58:14.019 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4500 IOs, 1840869376 bytes 2021-03-09T02:58:14.337 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4600 IOs, 1934664704 bytes 2021-03-09T02:58:14.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4700 IOs, 1967873024 bytes 2021-03-09T02:58:14.848 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4800 IOs, 1996633088 bytes 2021-03-09T02:58:15.104 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4900 IOs, 2039687168 bytes 2021-03-09T02:58:15.375 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5000 IOs, 2096202752 bytes 2021-03-09T02:58:15.660 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5100 IOs, 2120685568 bytes 2021-03-09T02:58:16.070 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5200 IOs, 2214563840 bytes 2021-03-09T02:58:16.336 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:5300 IOs, 2250017792 bytes 2021-03-09T02:58:16.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T02:58:16.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T02:58:16.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T02:58:16.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:58:16.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:58:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:16.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:58:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:58:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:58:16.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:58:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_status_in_pool_dir cluster1 mirror test up+replaying 2021-03-09T02:58:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:17.465 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:17.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:17.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:58:17.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local description_pattern= 2021-03-09T02:58:17.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local service_pattern= 2021-03-09T02:58:17.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.mirror_status 2021-03-09T02:58:17.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.mirror_status 2021-03-09T02:58:17.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:58:17.471 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:58:17.471 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ CEPH_ARGS= rbd --cluster cluster1 mirror image status mirror/test 2021-03-09T02:58:17.472 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ tee /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:58:17.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:test: 2021-03-09T02:58:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: global_id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:58:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+replaying 2021-03-09T02:58:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: replaying, {"bytes_per_second":17178555.73,"bytes_per_snapshot":523912192.0,"local_snapshot_timestamp":1615258671,"remote_snapshot_timestamp":1615258671,"replay_state":"idle"} 2021-03-09T02:58:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: service: mirror.1 on smithi144 2021-03-09T02:58:17.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:57:57 2021-03-09T02:58:17.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: peer_sites: 2021-03-09T02:58:17.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: name: cluster2 2021-03-09T02:58:17.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: state: up+stopped 2021-03-09T02:58:17.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: description: local image is primary 2021-03-09T02:58:17.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: last_update: 2021-03-09 02:57:58 2021-03-09T02:58:17.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ state: .*up+replaying /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:58:17.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: state: up+replaying 2021-03-09T02:58:17.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^ description: .* /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:58:17.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: description: replaying, {"bytes_per_second":17178555.73,"bytes_per_snapshot":523912192.0,"local_snapshot_timestamp":1615258671,"remote_snapshot_timestamp":1615258671,"replay_state":"idle"} 2021-03-09T02:58:17.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T02:58:17.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:58:17.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep ^up+ 2021-03-09T02:58:17.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:58:17.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep service: *mirror..* on /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:58:17.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: service: mirror.1 on smithi144 2021-03-09T02:58:17.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -nEe s/^ last_update: *(.*) *$/\1/p /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.mirror_status 2021-03-09T02:58:17.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update=2021-03-09 02:57:57 2021-03-09T02:58:17.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_mirror_pool_status_verbose cluster1 mirror test up+replaying 2021-03-09 02:57:57 2021-03-09T02:58:17.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:17.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:17.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:17.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state_pattern=up+replaying 2021-03-09T02:58:17.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local prev_last_update=2021-03-09 02:57:57 2021-03-09T02:58:17.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror.mirror_status 2021-03-09T02:58:17.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror.mirror_status 2021-03-09T02:58:17.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:58:17.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror.mirror_status 2021-03-09T02:58:17.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 mirror pool status mirror --verbose --format xml 2021-03-09T02:58:17.633 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd: failed to query services: (13) Permission denied 2021-03-09T02:58:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local last_update state 2021-03-09T02:58:17.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/last_update 2021-03-09T02:58:17.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ last_update=2021-03-09 02:57:57 2021-03-09T02:58:17.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //images/image[name='test']/state 2021-03-09T02:58:17.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ state=up+replaying 2021-03-09T02:58:17.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo up+replaying 2021-03-09T02:58:17.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep up+replaying 2021-03-09T02:58:17.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:up+replaying 2021-03-09T02:58:17.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:58:17.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:58:17.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap10 2021-03-09T02:58:17.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap10 2021-03-09T02:58:17.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:58:17.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:17.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:17.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap10 2021-03-09T02:58:17.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10 --debug-rbd=20 --debug-journaler=20 2021-03-09T02:58:30.037 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6805128512952907) > weight (0.25) 2021-03-09T02:58:30.037 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.34829914057806244) > weight (0.25) 2021-03-09T02:58:30.037 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.30395892945710945) > weight (0.25) 2021-03-09T02:58:30.038 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4968536124542743) > weight (0.25) 2021-03-09T02:58:30.038 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T02:58:48.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:58:48.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:48.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:48.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:48.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:58:48.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:48.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:48.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:48.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:58:48.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:58:48.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:58:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:58:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:49.493 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:58:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:58:49.494 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:58:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:58:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:58:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:58:49.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:49.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + local instance=cluster1 2021-03-09T02:58:49.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loop_instance 2021-03-09T02:58:49.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0 2021-03-09T02:58:49.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:grep -i state 2021-03-09T02:58:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:58:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:58:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:58:49.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:58:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:58:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:58:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:58:49.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:58:49.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:58:49.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:58:49.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:58:49.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:58:49.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:58:49.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:58:49.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:58:49.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:58:49.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:58:49.503 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:58:49.503 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:58:49.503 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:49.503 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:58:49.503 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:58:49.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:58:49.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:58:49.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:58:49.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:58:49.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:58:49.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:58:49.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:58:49.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:58:49.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:58:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:58:49.506 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:58:49.520 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:58:49.520 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:58:49.520 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:58:49.521 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:58:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:58:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:58:49.627 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:58:49.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:58:49.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:58:49.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:58:49.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:58:49.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:58:49.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:58:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:58:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:58:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:58:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:58:49.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:58:49.633 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:58:49.633 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:58:49.633 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:58:49.633 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:58:49.633 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:58:49.634 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:58:49.634 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:49.634 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:58:49.635 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:58:49.635 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:58:49.635 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:58:49.635 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:58:49.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:58:49.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:58:49.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:58:49.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:58:49.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:58:49.637 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:58:49.637 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:58:49.637 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:58:49.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:58:49.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:58:49.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:58:49.652 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:58:49.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:58:49.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:58:49.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo "state": "Replaying" 2021-03-09T02:58:49.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ grep -i Replaying 2021-03-09T02:58:49.762 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:58:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:58:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:58:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:58:49.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-03-09T02:58:49.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:58:49.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:58:49.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:49.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:49.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = journal ] 2021-03-09T02:58:49.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = snapshot ] 2021-03-09T02:58:49.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-03-09T02:58:49.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:58:49.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:58:49.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:49.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:49.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster2-mirror-test.status 2021-03-09T02:58:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster2-mirror-test.status 2021-03-09T02:58:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:58:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:58:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.status 2021-03-09T02:58:49.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.status 2021-03-09T02:58:49.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:58:49.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:58:49.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-03-09T02:58:49.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:58:49.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:49.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:49.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-03-09T02:58:50.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Snapshot ID: 42 2021-03-09T02:58:50.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:58:50.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:58:50.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:50.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:50.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:58:50.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-03-09T02:58:50.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:58:50.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-03-09T02:58:50.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snapshot_id=42 2021-03-09T02:58:50.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:58:50.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.2 2021-03-09T02:58:50.831 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:58:50.832 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:50.832 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:50.832 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:50.832 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:58:50.833 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:58:50.833 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:58:50.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:58:50.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=38 2021-03-09T02:58:50.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 42 = 38 2021-03-09T02:58:50.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.4 2021-03-09T02:58:51.313 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:58:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:58:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:58:51.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:58:51.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:58:51.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:58:51.393 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:58:51.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:58:51.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:58:51.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:58:51.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 42 = 2021-03-09T02:58:51.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.8 2021-03-09T02:58:52.196 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:58:52.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:52.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:52.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:52.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:58:52.198 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:58:52.198 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:58:52.273 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:58:52.274 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:58:52.280 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:58:52.281 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:58:52.281 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:58:52.281 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:58:52.281 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 42 = 2021-03-09T02:58:52.281 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1.6 2021-03-09T02:58:53.880 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:58:53.880 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:53.880 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:53.881 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:53.881 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:58:53.881 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:58:53.881 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:58:53.948 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:58:53.948 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:58:53.952 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:58:53.952 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:58:53.952 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:58:53.953 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:58:53.953 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 42 = 2021-03-09T02:58:53.954 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 2 2021-03-09T02:58:55.955 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:58:55.956 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:55.956 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:55.956 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:55.957 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:58:55.957 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:58:55.957 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:58:56.032 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:58:56.038 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=42 2021-03-09T02:58:56.038 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 42 = 42 2021-03-09T02:58:56.038 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:58:56.038 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snap_present cluster1 mirror test snap10 2021-03-09T02:58:56.039 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:58:56.039 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:56.039 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:56.039 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap10 2021-03-09T02:58:56.040 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:58:56.040 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:58:57.039 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 info mirror/test@snap10 2021-03-09T02:58:57.102 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:rbd image 'test': 2021-03-09T02:58:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: size 512 MiB in 128 objects 2021-03-09T02:58:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: order 22 (4 MiB objects) 2021-03-09T02:58:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: snapshot_count: 20 2021-03-09T02:58:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: id: 10a0f27e4330 2021-03-09T02:58:57.103 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: block_name_prefix: rbd_data.10a0f27e4330 2021-03-09T02:58:57.104 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: format: 2 2021-03-09T02:58:57.104 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: features: layering, exclusive-lock, non-primary 2021-03-09T02:58:57.104 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: op_features: 2021-03-09T02:58:57.104 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: flags: 2021-03-09T02:58:57.105 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: create_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:58:57.105 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: access_timestamp: Tue Mar 9 02:57:58 2021 2021-03-09T02:58:57.105 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: modify_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:58:57.106 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: protected: False 2021-03-09T02:58:57.106 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring state: enabled 2021-03-09T02:58:57.106 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring mode: snapshot 2021-03-09T02:58:57.107 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring global id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:58:57.107 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring primary: false 2021-03-09T02:58:57.108 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:58:57.109 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snap9-clean ] 2021-03-09T02:58:57.109 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap9-clean 2021-03-09T02:58:57.110 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:57.110 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:57.111 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap9-clean 2021-03-09T02:58:57.111 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:58:57.111 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:58:57.112 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:58:57.112 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:58:57.113 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:58:57.872 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:58:57.876 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:58:58.726 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:58:58.731 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:58:59.005 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:58:59.132 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:58:59.133 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap10 2021-03-09T02:58:59.133 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:58:59.133 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:58:59.133 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap10 2021-03-09T02:58:59.133 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:58:59.134 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:58:59.134 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:58:59.134 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:58:59.135 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:58:59.809 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:58:59.813 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:00.653 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:00.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:00.933 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:01.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:01.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ clean_snap_name=snap10-clean 2021-03-09T02:59:01.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_snap cluster2 mirror test snap10-clean 2021-03-09T02:59:01.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:01.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:01.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:01.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap10-clean 2021-03-09T02:59:01.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror snap create test@snap10-clean --debug-rbd=20 --debug-journaler=20 2021-03-09T02:59:01.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_started cluster1 mirror test 2021-03-09T02:59:01.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:59:01.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:01.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:01.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_replay_state cluster1 mirror test started 2021-03-09T02:59:01.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:59:01.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=started 2021-03-09T02:59:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:59:01.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:59:02.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_replay_state cluster1 mirror test started 2021-03-09T02:59:02.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:59:02.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:02.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:02.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=started 2021-03-09T02:59:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_result 2021-03-09T02:59:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=stopped 2021-03-09T02:59:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemons cluster1 rbd mirror status mirror/test 2021-03-09T02:59:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_instance=cluster1 2021-03-09T02:59:02.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + shift 2021-03-09T02:59:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:59:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance=cluster1grep 2021-03-09T02:59:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: -i state+ 2021-03-09T02:59:02.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:local loop_instance 2021-03-09T02:59:02.419 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0 2021-03-09T02:59:02.421 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ cluster1 != cluster1 ] 2021-03-09T02:59:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 0 3 2021-03-09T02:59:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:0 rbd mirror status mirror/test 2021-03-09T02:59:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:0 2021-03-09T02:59:02.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:59:02.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:59:02.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 cluster instance 2021-03-09T02:59:02.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:59:02.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:59:02.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:59:02.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:59:02.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:59:02.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:59:02.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:59:02.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:59:02.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:59:02.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:59:02.427 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:0 cluster1 2021-03-09T02:59:02.427 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:0 2021-03-09T02:59:02.427 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:59:02.428 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:59:02.428 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:0 local_cluster instance 2021-03-09T02:59:02.428 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:0 2021-03-09T02:59:02.429 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:59:02.429 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:59:02.429 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:59:02.430 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:59:02.430 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 0 = cluster1:0 ] 2021-03-09T02:59:02.430 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:59:02.431 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:59:02.431 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=0 2021-03-09T02:59:02.432 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=0 2021-03-09T02:59:02.432 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.0 admin socket 2021-03-09T02:59:02.440 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:59:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:59:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok 2021-03-09T02:59:02.441 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.0.asok rbd mirror status mirror/test 2021-03-09T02:59:02.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:Invalid command: unused arguments: ['mirror/test'] 2021-03-09T02:59:02.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd mirror status : get status for rbd mirror 2021-03-09T02:59:02.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:admin_socket: invalid command 2021-03-09T02:59:02.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ admin_daemon cluster1:1 rbd mirror status mirror/test 2021-03-09T02:59:02.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1:1 2021-03-09T02:59:02.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T02:59:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:59:02.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 cluster instance 2021-03-09T02:59:02.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:59:02.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=cluster 2021-03-09T02:59:02.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:59:02.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:59:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:59:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:59:02.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval cluster=cluster1 2021-03-09T02:59:02.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:59:02.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:59:02.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:59:02.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ daemon_asok_file cluster1:1 cluster1 2021-03-09T02:59:02.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1:1 2021-03-09T02:59:02.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:59:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance 2021-03-09T02:59:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set_cluster_instance cluster1:1 local_cluster instance 2021-03-09T02:59:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local val=cluster1:1 2021-03-09T02:59:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster_var_name=local_cluster 2021-03-09T02:59:02.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local instance_var_name=instance 2021-03-09T02:59:02.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cluster=cluster1 2021-03-09T02:59:02.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:59:02.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 1 = cluster1:1 ] 2021-03-09T02:59:02.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval local_cluster=cluster1 2021-03-09T02:59:02.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local_cluster=cluster1 2021-03-09T02:59:02.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ eval instance=1 2021-03-09T02:59:02.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ instance=1 2021-03-09T02:59:02.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph-conf --cluster cluster1 --name client.mirror.1 admin socket 2021-03-09T02:59:02.572 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:59:02.572 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local asok_file=/var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:59:02.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -S /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok 2021-03-09T02:59:02.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ ceph --admin-daemon /var/run/ceph/rbd-mirror.cluster1-client.mirror.1.asok rbd mirror status mirror/test 2021-03-09T02:59:02.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:02.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ status_result= "state": "Replaying" 2021-03-09T02:59:02.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ + echo "state": "Replaying" 2021-03-09T02:59:02.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:grep -i Replaying 2021-03-09T02:59:02.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: "state": "Replaying" 2021-03-09T02:59:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=started 2021-03-09T02:59:02.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test started = started 2021-03-09T02:59:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_replay_complete cluster1 cluster2 mirror test 2021-03-09T02:59:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:59:02.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = journal ] 2021-03-09T02:59:02.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ snapshot = snapshot ] 2021-03-09T02:59:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snapshot_sync_complete cluster1 cluster2 mirror test 2021-03-09T02:59:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_cluster=cluster1 2021-03-09T02:59:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:02.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster2-mirror-test.status 2021-03-09T02:59:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster2-mirror-test.status 2021-03-09T02:59:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:59:02.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local status_log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:59:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mkfname cluster1-mirror-test.status 2021-03-09T02:59:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo cluster1-mirror-test.status 2021-03-09T02:59:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -e s|[/ ]|_|g 2021-03-09T02:59:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local local_status_log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:59:02.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ mirror_image_snapshot cluster2 mirror test 2021-03-09T02:59:02.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:02.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:02.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:02.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 mirror image snapshot mirror/test 2021-03-09T02:59:03.433 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Snapshot ID: 45 2021-03-09T02:59:03.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster2 mirror test /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:59:03.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:03.463 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:03.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:03.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.status 2021-03-09T02:59:03.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap list --all mirror/test --format xml 2021-03-09T02:59:03.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:59:03.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/id 2021-03-09T02:59:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snapshot_id=45 2021-03-09T02:59:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:59:03.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.2 2021-03-09T02:59:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:59:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:59:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:03.733 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:59:03.733 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:59:03.733 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:59:03.809 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:59:03.814 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=42 2021-03-09T02:59:03.815 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 45 = 42 2021-03-09T02:59:03.815 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.4 2021-03-09T02:59:04.216 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:59:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:59:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:59:04.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:59:04.218 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:59:04.297 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:59:04.409 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:59:04.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:59:04.411 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:59:04.411 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:59:04.411 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:59:04.411 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 45 = 2021-03-09T02:59:04.412 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.8 2021-03-09T02:59:05.104 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:59:05.105 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:59:05.105 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:05.105 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:05.105 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:59:05.105 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:59:05.106 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:59:05.184 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ true 2021-03-09T02:59:05.184 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:59:05.188 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:-:1.1: Document is empty 2021-03-09T02:59:05.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: 2021-03-09T02:59:05.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:^ 2021-03-09T02:59:05.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id= 2021-03-09T02:59:05.190 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 45 = 2021-03-09T02:59:05.190 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1.6 2021-03-09T02:59:06.791 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_newest_mirror_snapshot cluster1 mirror test /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:59:06.791 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:59:06.791 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:06.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:06.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local log=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.status 2021-03-09T02:59:06.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 snap list --all mirror/test --format xml 2021-03-09T02:59:06.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -c //snapshots/snapshot[namespace/complete='true' and position()=last()] 2021-03-09T02:59:06.867 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ xmlstarlet sel -t -v //snapshot/namespace/primary_snap_id 2021-03-09T02:59:06.872 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local primary_snapshot_id=45 2021-03-09T02:59:06.873 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test 45 = 45 2021-03-09T02:59:06.873 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:06.873 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_snap_present cluster1 mirror test snap10-clean 2021-03-09T02:59:06.873 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T02:59:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap10-clean 2021-03-09T02:59:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T02:59:06.874 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T02:59:07.873 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 info mirror/test@snap10-clean 2021-03-09T02:59:07.946 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:rbd image 'test': 2021-03-09T02:59:07.946 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: size 512 MiB in 128 objects 2021-03-09T02:59:07.946 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: order 22 (4 MiB objects) 2021-03-09T02:59:07.947 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: snapshot_count: 21 2021-03-09T02:59:07.947 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: id: 10a0f27e4330 2021-03-09T02:59:07.947 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: block_name_prefix: rbd_data.10a0f27e4330 2021-03-09T02:59:07.947 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: format: 2 2021-03-09T02:59:07.948 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: features: layering, exclusive-lock, non-primary 2021-03-09T02:59:07.948 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: op_features: 2021-03-09T02:59:07.948 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: flags: 2021-03-09T02:59:07.948 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: create_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:59:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: access_timestamp: Tue Mar 9 02:58:59 2021 2021-03-09T02:59:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: modify_timestamp: Tue Mar 9 02:50:00 2021 2021-03-09T02:59:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: protected: False 2021-03-09T02:59:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring state: enabled 2021-03-09T02:59:07.949 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring mode: snapshot 2021-03-09T02:59:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring global id: 18a549b3-61f2-46e5-af4d-1a9b02c6e0ad 2021-03-09T02:59:07.950 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout: mirroring primary: false 2021-03-09T02:59:07.951 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:07.952 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 1 10 2021-03-09T02:59:07.953 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap1 2021-03-09T02:59:07.953 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap1 2021-03-09T02:59:07.953 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:07.953 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:07.954 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap1 2021-03-09T02:59:07.954 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:07.954 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:07.954 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:07.955 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:07.955 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:08.643 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:08.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:09.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:09.508 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:09.789 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:09.916 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:09.917 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap1-clean 2021-03-09T02:59:09.917 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap1-clean 2021-03-09T02:59:09.917 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:09.917 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:09.918 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap1-clean 2021-03-09T02:59:09.918 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:09.918 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:09.918 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:09.919 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:09.919 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap1-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:10.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:10.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap1-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:11.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:11.510 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:11.788 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:11.917 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:11.917 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap2 2021-03-09T02:59:11.918 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap2 2021-03-09T02:59:11.918 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:11.918 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:11.918 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap2 2021-03-09T02:59:11.918 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:11.919 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:11.919 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:11.919 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:11.920 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:12.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:12.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:13.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:13.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:13.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:13.835 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:13.835 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap2-clean 2021-03-09T02:59:13.836 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap2-clean 2021-03-09T02:59:13.836 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:13.836 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:13.836 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap2-clean 2021-03-09T02:59:13.836 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:13.837 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:13.837 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:13.837 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:13.838 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap2-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:14.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:14.488 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap2-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:15.340 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:15.343 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:15.624 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:15.751 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap3 2021-03-09T02:59:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap3 2021-03-09T02:59:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:15.752 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap3 2021-03-09T02:59:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:15.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:15.754 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:16.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:16.427 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:17.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:17.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:17.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:17.727 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:17.727 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap3-clean 2021-03-09T02:59:17.727 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap3-clean 2021-03-09T02:59:17.728 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:17.728 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:17.728 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap3-clean 2021-03-09T02:59:17.728 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:17.728 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:17.729 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:17.729 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:17.729 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap3-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:18.444 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:18.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap3-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:19.272 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:19.275 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:19.555 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:19.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:19.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap4 2021-03-09T02:59:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap4 2021-03-09T02:59:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap4 2021-03-09T02:59:19.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:19.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:19.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:19.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:19.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:20.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:20.362 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:21.283 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:21.286 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:21.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:21.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:21.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap4-clean 2021-03-09T02:59:21.692 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap4-clean 2021-03-09T02:59:21.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:21.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:21.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap4-clean 2021-03-09T02:59:21.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:21.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:21.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:21.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:21.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap4-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:22.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:22.363 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap4-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:23.266 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:23.270 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:23.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:23.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:23.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap5 2021-03-09T02:59:23.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap5 2021-03-09T02:59:23.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:23.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:23.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap5 2021-03-09T02:59:23.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:23.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:23.679 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:23.679 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:23.679 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:24.349 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:24.352 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:25.282 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:25.287 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:25.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:25.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:25.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap5-clean 2021-03-09T02:59:25.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap5-clean 2021-03-09T02:59:25.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:25.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:25.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap5-clean 2021-03-09T02:59:25.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:25.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:25.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:25.701 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:25.701 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap5-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:26.351 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:26.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap5-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:27.202 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:27.205 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:27.486 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:27.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap6 2021-03-09T02:59:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap6 2021-03-09T02:59:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:27.616 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:27.617 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap6 2021-03-09T02:59:27.617 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:27.617 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:27.617 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:27.618 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:27.618 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:28.267 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:28.271 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:29.151 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:29.155 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:29.431 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:29.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:29.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap6-clean 2021-03-09T02:59:29.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap6-clean 2021-03-09T02:59:29.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:29.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:29.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap6-clean 2021-03-09T02:59:29.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:29.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:29.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:29.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:29.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap6-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:30.246 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:30.251 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap6-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:31.135 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:31.138 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:31.416 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:31.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:31.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap7 2021-03-09T02:59:31.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap7 2021-03-09T02:59:31.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:31.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:31.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap7 2021-03-09T02:59:31.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:31.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:31.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:32.207 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:32.211 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:33.128 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:33.133 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:33.412 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:33.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:33.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap7-clean 2021-03-09T02:59:33.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap7-clean 2021-03-09T02:59:33.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:33.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:33.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap7-clean 2021-03-09T02:59:33.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:33.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:33.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:33.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:33.538 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap7-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:34.200 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:34.204 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap7-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:35.083 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:35.086 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:35.371 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:35.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:35.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap8 2021-03-09T02:59:35.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap8 2021-03-09T02:59:35.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:35.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:35.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap8 2021-03-09T02:59:35.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:35.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:35.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:35.501 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:36.187 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:36.190 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:37.112 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:37.116 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:37.391 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:37.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:37.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap8-clean 2021-03-09T02:59:37.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap8-clean 2021-03-09T02:59:37.524 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:37.524 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:37.524 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap8-clean 2021-03-09T02:59:37.524 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:37.524 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:37.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:37.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:37.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap8-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:38.179 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:38.183 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap8-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:39.096 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:39.101 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:39.382 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:39.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap9 2021-03-09T02:59:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap9 2021-03-09T02:59:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:39.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap9 2021-03-09T02:59:39.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:39.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:39.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:39.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:39.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:40.212 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:40.215 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:41.094 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:41.100 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:41.381 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:41.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:41.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap9-clean 2021-03-09T02:59:41.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap9-clean 2021-03-09T02:59:41.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:41.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:41.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap9-clean 2021-03-09T02:59:41.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:41.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:41.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:41.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:41.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap9-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:42.188 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:42.192 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap9-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:43.110 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:43.115 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:43.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:43.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:43.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap10 2021-03-09T02:59:43.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap10 2021-03-09T02:59:43.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:43.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:43.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap10 2021-03-09T02:59:43.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:43.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:43.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:43.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:43.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10 /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:44.209 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:44.213 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10 /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:45.053 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:45.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:45.339 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap10-clean 2021-03-09T02:59:45.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ compare_image_snaps mirror test snap10-clean 2021-03-09T02:59:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap_name=snap10-clean 2021-03-09T02:59:45.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local ret=0 2021-03-09T02:59:45.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local rmt_export=/tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:45.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local loc_export=/tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:45.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:45.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 -p mirror export test@snap10-clean /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export 2021-03-09T02:59:46.144 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:46.148 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster1 -p mirror export test@snap10-clean /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:46.246 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6403858456813599) > weight (0.25) 2021-03-09T02:59:46.247 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8007364838783821) > weight (0.25) 2021-03-09T02:59:46.247 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8345334279929596) > weight (0.25) 2021-03-09T02:59:46.247 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8497077889952507) > weight (0.25) 2021-03-09T02:59:46.248 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T02:59:47.024 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Exporting image: 1% complete... Exporting image: 2% complete... Exporting image: 3% complete... Exporting image: 4% complete... Exporting image: 5% complete... Exporting image: 6% complete... Exporting image: 7% complete... Exporting image: 8% complete... Exporting image: 9% complete... Exporting image: 10% complete... Exporting image: 11% complete... Exporting image: 12% complete... Exporting image: 13% complete... Exporting image: 14% complete... Exporting image: 15% complete... Exporting image: 16% complete... Exporting image: 17% complete... Exporting image: 18% complete... Exporting image: 19% complete... Exporting image: 20% complete... Exporting image: 21% complete... Exporting image: 22% complete... Exporting image: 23% complete... Exporting image: 24% complete... Exporting image: 25% complete... Exporting image: 26% complete... Exporting image: 27% complete... Exporting image: 28% complete... Exporting image: 29% complete... Exporting image: 30% complete... Exporting image: 31% complete... Exporting image: 32% complete... Exporting image: 33% complete... Exporting image: 34% complete... Exporting image: 35% complete... Exporting image: 36% complete... Exporting image: 37% complete... Exporting image: 38% complete... Exporting image: 39% complete... Exporting image: 40% complete... Exporting image: 41% complete... Exporting image: 42% complete... Exporting image: 43% complete... Exporting image: 44% complete... Exporting image: 45% complete... Exporting image: 46% complete... Exporting image: 47% complete... Exporting image: 48% complete... Exporting image: 49% complete... Exporting image: 50% complete... Exporting image: 51% complete... Exporting image: 52% complete... Exporting image: 53% complete... Exporting image: 54% complete... Exporting image: 55% complete... Exporting image: 56% complete... Exporting image: 57% complete... Exporting image: 58% complete... Exporting image: 59% complete... Exporting image: 60% complete... Exporting image: 61% complete... Exporting image: 62% complete... Exporting image: 63% complete... Exporting image: 64% complete... Exporting image: 65% complete... Exporting image: 66% complete... Exporting image: 67% complete... Exporting image: 68% complete... Exporting image: 69% complete... Exporting image: 70% complete... Exporting image: 71% complete... Exporting image: 72% complete... Exporting image: 73% complete... Exporting image: 74% complete... Exporting image: 75% complete... Exporting image: 76% complete... Exporting image: 77% complete... Exporting image: 78% complete... Exporting image: 79% complete... Exporting image: 80% complete... Exporting image: 81% complete... Exporting image: 82% complete... Exporting image: 83% complete... Exporting image: 84% complete... Exporting image: 85% complete... Exporting image: 86% complete... Exporting image: 87% complete... Exporting image: 88% complete... Exporting image: 89% complete... Exporting image: 90% complete... Exporting image: 91% complete... Exporting image: 92% complete... Exporting image: 93% complete... Exporting image: 94% complete... Exporting image: 95% complete... Exporting image: 96% complete... Exporting image: 97% complete... Exporting image: 98% complete... Exporting image: 99% complete... Exporting image: 100% complete...done. 2021-03-09T02:59:47.029 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ cmp /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:47.310 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rm -f /tmp/tmp.y6RrY8NxKl/cluster2-mirror-test.export /tmp/tmp.y6RrY8NxKl/cluster1-mirror-test.export 2021-03-09T02:59:47.440 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T02:59:47.441 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 1 10 2021-03-09T02:59:47.442 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap1 2021-03-09T02:59:47.442 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap1 2021-03-09T02:59:47.442 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:47.442 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:47.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:47.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap1 2021-03-09T02:59:47.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1 2021-03-09T02:59:48.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:48.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap1-clean 2021-03-09T02:59:48.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap1-clean 2021-03-09T02:59:48.478 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:48.479 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:48.479 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:48.479 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap1-clean 2021-03-09T02:59:48.479 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap1-clean 2021-03-09T02:59:49.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:49.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap2 2021-03-09T02:59:49.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap2 2021-03-09T02:59:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap2 2021-03-09T02:59:49.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2 2021-03-09T02:59:50.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:50.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap2-clean 2021-03-09T02:59:50.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap2-clean 2021-03-09T02:59:50.486 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:50.486 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:50.486 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:50.486 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap2-clean 2021-03-09T02:59:50.486 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap2-clean 2021-03-09T02:59:51.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:51.508 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap3 2021-03-09T02:59:51.508 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap3 2021-03-09T02:59:51.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:51.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:51.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:51.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap3 2021-03-09T02:59:51.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3 2021-03-09T02:59:52.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:52.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap3-clean 2021-03-09T02:59:52.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap3-clean 2021-03-09T02:59:52.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:52.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:52.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:52.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap3-clean 2021-03-09T02:59:52.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap3-clean 2021-03-09T02:59:53.411 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:53.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap4 2021-03-09T02:59:53.450 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap4 2021-03-09T02:59:53.450 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:53.450 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:53.450 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:53.451 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap4 2021-03-09T02:59:53.451 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4 2021-03-09T02:59:54.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:54.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap4-clean 2021-03-09T02:59:54.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap4-clean 2021-03-09T02:59:54.517 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:54.517 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:54.517 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:54.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap4-clean 2021-03-09T02:59:54.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap4-clean 2021-03-09T02:59:55.510 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:55.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap5 2021-03-09T02:59:55.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap5 2021-03-09T02:59:55.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:55.538 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:55.538 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:55.538 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap5 2021-03-09T02:59:55.538 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5 2021-03-09T02:59:56.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:56.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap5-clean 2021-03-09T02:59:56.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap5-clean 2021-03-09T02:59:56.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:56.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:56.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:56.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap5-clean 2021-03-09T02:59:56.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap5-clean 2021-03-09T02:59:57.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:57.538 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap6 2021-03-09T02:59:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap6 2021-03-09T02:59:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:57.539 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap6 2021-03-09T02:59:57.540 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6 2021-03-09T02:59:58.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:58.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap6-clean 2021-03-09T02:59:58.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap6-clean 2021-03-09T02:59:58.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:58.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:58.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:58.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap6-clean 2021-03-09T02:59:58.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap6-clean 2021-03-09T02:59:59.542 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T02:59:59.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap7 2021-03-09T02:59:59.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap7 2021-03-09T02:59:59.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T02:59:59.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T02:59:59.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T02:59:59.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap7 2021-03-09T02:59:59.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7 2021-03-09T03:00:00.541 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T03:00:00.558 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap7-clean 2021-03-09T03:00:00.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap7-clean 2021-03-09T03:00:00.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:00.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:00.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:00.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap7-clean 2021-03-09T03:00:00.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap7-clean 2021-03-09T03:00:01.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T03:00:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap8 2021-03-09T03:00:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap8 2021-03-09T03:00:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:01.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap8 2021-03-09T03:00:01.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8 2021-03-09T03:00:02.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T03:00:02.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap8-clean 2021-03-09T03:00:02.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap8-clean 2021-03-09T03:00:02.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:02.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:02.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:02.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap8-clean 2021-03-09T03:00:02.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap8-clean 2021-03-09T03:00:03.415 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T03:00:03.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap9 2021-03-09T03:00:03.448 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap9 2021-03-09T03:00:03.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:03.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:03.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:03.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap9 2021-03-09T03:00:03.450 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9 2021-03-09T03:00:04.841 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T03:00:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap9-clean 2021-03-09T03:00:04.860 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap9-clean 2021-03-09T03:00:04.861 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:04.861 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:04.861 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:04.861 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap9-clean 2021-03-09T03:00:04.862 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap9-clean 2021-03-09T03:00:05.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T03:00:05.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap10 2021-03-09T03:00:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap10 2021-03-09T03:00:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap10 2021-03-09T03:00:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10 2021-03-09T03:00:06.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T03:00:06.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap10-clean 2021-03-09T03:00:06.623 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_snapshot cluster2 mirror test snap10-clean 2021-03-09T03:00:06.624 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:06.624 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:06.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:06.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local snap=snap10-clean 2021-03-09T03:00:06.626 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 snap rm mirror/test@snap10-clean 2021-03-09T03:00:07.623 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing snap: 100% complete...done. 2021-03-09T03:00:07.646 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_image_retry cluster2 mirror test 2021-03-09T03:00:07.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:07.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:07.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:07.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0 2021-03-09T03:00:07.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ remove_image cluster2 mirror test 2021-03-09T03:00:07.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:07.649 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:07.649 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:07.649 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 snap purge mirror/test 2021-03-09T03:00:07.703 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 rm mirror/test 2021-03-09T03:00:09.211 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr: Removing image: 1% complete... Removing image: 2% complete... Removing image: 3% complete... Removing image: 4% complete... Removing image: 5% complete... Removing image: 6% complete... Removing image: 7% complete... Removing image: 8% complete... Removing image: 9% complete... Removing image: 10% complete... Removing image: 11% complete... Removing image: 12% complete... Removing image: 13% complete... Removing image: 14% complete... Removing image: 15% complete... Removing image: 16% complete... Removing image: 17% complete... Removing image: 18% complete... Removing image: 19% complete... Removing image: 20% complete... Removing image: 21% complete... Removing image: 22% complete... Removing image: 23% complete... Removing image: 24% complete... Removing image: 25% complete... Removing image: 26% complete... Removing image: 27% complete... Removing image: 28% complete... Removing image: 29% complete... Removing image: 30% complete... Removing image: 31% complete... Removing image: 32% complete... Removing image: 33% complete... Removing image: 34% complete... Removing image: 35% complete... Removing image: 36% complete... Removing image: 37% complete... Removing image: 38% complete... Removing image: 39% complete... Removing image: 40% complete... Removing image: 41% complete... Removing image: 42% complete... Removing image: 43% complete... Removing image: 44% complete... Removing image: 45% complete... Removing image: 46% complete... Removing image: 47% complete... Removing image: 48% complete... Removing image: 49% complete... Removing image: 50% complete... Removing image: 51% complete... Removing image: 52% complete... Removing image: 53% complete... Removing image: 54% complete... Removing image: 55% complete... Removing image: 56% complete... Removing image: 57% complete... Removing image: 58% complete... Removing image: 59% complete... Removing image: 60% complete... Removing image: 61% complete... Removing image: 62% complete... Removing image: 63% complete... Removing image: 64% complete... Removing image: 65% complete... Removing image: 66% complete... Removing image: 67% complete... Removing image: 68% complete... Removing image: 69% complete... Removing image: 70% complete... Removing image: 71% complete... Removing image: 72% complete... Removing image: 73% complete... Removing image: 74% complete... Removing image: 75% complete... Removing image: 76% complete... Removing image: 77% complete... Removing image: 78% complete... Removing image: 79% complete... Removing image: 80% complete... Removing image: 81% complete... Removing image: 82% complete... Removing image: 83% complete... Removing image: 84% complete... Removing image: 85% complete... Removing image: 86% complete... Removing image: 87% complete... Removing image: 88% complete... Removing image: 89% complete... Removing image: 90% complete... Removing image: 91% complete... Removing image: 92% complete... Removing image: 93% complete... Removing image: 94% complete... Removing image: 95% complete... Removing image: 96% complete... Removing image: 97% complete... Removing image: 98% complete... Removing image: 99% complete... Removing image: 100% complete...done. 2021-03-09T03:00:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:00:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ wait_for_image_present cluster1 mirror test deleted 2021-03-09T03:00:09.216 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:09.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:09.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:09.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local state=deleted 2021-03-09T03:00:09.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image_id= 2021-03-09T03:00:09.217 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local s 2021-03-09T03:00:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 2021-03-09T03:00:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_image_id cluster1 mirror test 2021-03-09T03:00:09.218 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:09.219 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:09.219 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:09.219 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-03-09T03:00:09.219 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-03-09T03:00:09.283 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image_id=10a0f27e4330 2021-03-09T03:00:09.284 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 0.1 2021-03-09T03:00:09.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_present cluster1 mirror test deleted 10a0f27e4330 2021-03-09T03:00:09.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:09.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=deleted 2021-03-09T03:00:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image_id=10a0f27e4330 2021-03-09T03:00:09.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=deleted 2021-03-09T03:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_image_id 2021-03-09T03:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_image_id cluster1 mirror test 2021-03-09T03:00:09.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:09.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-03-09T03:00:09.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-03-09T03:00:09.455 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_image_id=10a0f27e4330 2021-03-09T03:00:09.455 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 10a0f27e4330 2021-03-09T03:00:09.455 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -z 10a0f27e4330 -o 10a0f27e4330 = 10a0f27e4330 2021-03-09T03:00:09.455 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=present 2021-03-09T03:00:09.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test deleted = present 2021-03-09T03:00:09.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 1 2021-03-09T03:00:10.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_present cluster1 mirror test deleted 10a0f27e4330 2021-03-09T03:00:10.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:10.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:10.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:10.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=deleted 2021-03-09T03:00:10.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image_id=10a0f27e4330 2021-03-09T03:00:10.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=deleted 2021-03-09T03:00:10.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_image_id 2021-03-09T03:00:10.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_image_id cluster1 mirror test 2021-03-09T03:00:10.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:10.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:10.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:10.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-03-09T03:00:10.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-03-09T03:00:10.531 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_image_id=10a0f27e4330 2021-03-09T03:00:10.531 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 10a0f27e4330 2021-03-09T03:00:10.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -z 10a0f27e4330 -o 10a0f27e4330 = 10a0f27e4330 2021-03-09T03:00:10.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=present 2021-03-09T03:00:10.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test deleted = present 2021-03-09T03:00:10.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 2 2021-03-09T03:00:11.042 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6047236179889545) > weight (0.25) 2021-03-09T03:00:11.042 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.936780158803074) > weight (0.25) 2021-03-09T03:00:11.043 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9555684832326603) > weight (0.25) 2021-03-09T03:00:11.043 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T03:00:11.044 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T03:00:11.045 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T03:00:11.046 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T03:00:11.037+0000 7f5ff090a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 20133) UID: 1000 2021-03-09T03:00:11.246 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:00:12.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_present cluster1 mirror test deleted 10a0f27e4330 2021-03-09T03:00:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:12.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=deleted 2021-03-09T03:00:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image_id=10a0f27e4330 2021-03-09T03:00:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=deleted 2021-03-09T03:00:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_image_id 2021-03-09T03:00:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_image_id cluster1 mirror test 2021-03-09T03:00:12.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:12.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:12.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:12.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-03-09T03:00:12.542 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-03-09T03:00:12.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_image_id=10a0f27e4330 2021-03-09T03:00:12.754 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 10a0f27e4330 2021-03-09T03:00:12.754 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -z 10a0f27e4330 -o 10a0f27e4330 = 10a0f27e4330 2021-03-09T03:00:12.755 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=present 2021-03-09T03:00:12.755 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test deleted = present 2021-03-09T03:00:12.756 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 4 2021-03-09T03:00:15.047 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T03:00:15.047 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:00:15.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:00:15.048 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T03:00:15.049 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T03:00:15.049 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T03:00:15.053 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T03:00:15.053 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T03:00:16.756 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_present cluster1 mirror test deleted 10a0f27e4330 2021-03-09T03:00:16.757 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:16.757 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:16.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:16.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=deleted 2021-03-09T03:00:16.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image_id=10a0f27e4330 2021-03-09T03:00:16.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=deleted 2021-03-09T03:00:16.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_image_id 2021-03-09T03:00:16.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_image_id cluster1 mirror test 2021-03-09T03:00:16.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:16.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:16.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:16.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-03-09T03:00:16.762 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-03-09T03:00:16.824 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_image_id=10a0f27e4330 2021-03-09T03:00:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 10a0f27e4330 2021-03-09T03:00:16.825 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -z 10a0f27e4330 -o 10a0f27e4330 = 10a0f27e4330 2021-03-09T03:00:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=present 2021-03-09T03:00:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test deleted = present 2021-03-09T03:00:16.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 8 2021-03-09T03:00:24.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_present cluster1 mirror test deleted 10a0f27e4330 2021-03-09T03:00:24.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:24.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=deleted 2021-03-09T03:00:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image_id=10a0f27e4330 2021-03-09T03:00:24.828 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=deleted 2021-03-09T03:00:24.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_image_id 2021-03-09T03:00:24.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_image_id cluster1 mirror test 2021-03-09T03:00:24.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:24.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:24.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:24.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-03-09T03:00:24.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-03-09T03:00:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_image_id=10a0f27e4330 2021-03-09T03:00:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 10a0f27e4330 2021-03-09T03:00:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -z 10a0f27e4330 -o 10a0f27e4330 = 10a0f27e4330 2021-03-09T03:00:24.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_state=present 2021-03-09T03:00:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test deleted = present 2021-03-09T03:00:24.908 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sleep 8 2021-03-09T03:00:32.909 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test_image_present cluster1 mirror test deleted 10a0f27e4330 2021-03-09T03:00:32.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:32.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:32.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:32.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local test_state=deleted 2021-03-09T03:00:32.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image_id=10a0f27e4330 2021-03-09T03:00:32.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_state=deleted 2021-03-09T03:00:32.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local current_image_id 2021-03-09T03:00:32.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ get_image_id cluster1 mirror test 2021-03-09T03:00:32.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster1 2021-03-09T03:00:32.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:32.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=test 2021-03-09T03:00:32.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster1 info mirror/test 2021-03-09T03:00:32.913 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ sed -ne s/^.*block_name_prefix: rbd_data\.//p 2021-03-09T03:00:32.951 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:rbd: error opening image test: (2) No such file or directory 2021-03-09T03:00:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ current_image_id= 2021-03-09T03:00:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test -n 2021-03-09T03:00:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ test deleted = deleted 2021-03-09T03:00:32.956 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:00:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ testlog TEST: create many images 2021-03-09T03:00:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ tee -a /tmp/tmp.y6RrY8NxKl/rbd-mirror.test.log 2021-03-09T03:00:32.957 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ date +%F %T 2021-03-09T03:00:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ echo 2021-03-09 03:00:32 TEST: create many images 2021-03-09T03:00:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:2021-03-09 03:00:32 TEST: create many images 2021-03-09T03:00:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ snap_name=snap 2021-03-09T03:00:32.959 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ seq 1 50 2021-03-09T03:00:32.960 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_1 2021-03-09T03:00:32.961 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_1 snapshot 128M 2021-03-09T03:00:32.961 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:32.961 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:32.962 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:32.966 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:32.966 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_1 2021-03-09T03:00:32.966 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:32.966 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:00:32.967 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:00:32.967 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:32.967 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_1 128M 2021-03-09T03:00:32.967 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:32.967 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:32.968 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:32.968 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:32.968 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_1 2021-03-09T03:00:32.968 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:32.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:00:32.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:00:32.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:00:32.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:32.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_1 2021-03-09T03:00:33.037 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:00:33.037 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_1 snapshot 2021-03-09T03:00:33.037 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:33.037 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:33.038 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_1 2021-03-09T03:00:33.038 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:00:33.038 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_1 snapshot 2021-03-09T03:00:33.506 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:00:33.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:00:33.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_1 2021-03-09T03:00:33.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:33.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:33.526 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_1 2021-03-09T03:00:33.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:00:33.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=11 2021-03-09T03:00:33.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:00:33.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_1 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:00:33.851 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 41119744 bytes 2021-03-09T03:00:34.178 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 109498368 bytes 2021-03-09T03:00:34.361 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 110135296 bytes 2021-03-09T03:00:34.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 142602240 bytes 2021-03-09T03:00:35.029 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 176738304 bytes 2021-03-09T03:00:35.318 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 228404224 bytes 2021-03-09T03:00:35.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 230920192 bytes 2021-03-09T03:00:35.813 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 302295040 bytes 2021-03-09T03:00:36.118 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 353936384 bytes 2021-03-09T03:00:36.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 398381056 bytes 2021-03-09T03:00:36.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 403680256 bytes 2021-03-09T03:00:36.850 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 441090048 bytes 2021-03-09T03:00:37.095 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 473798656 bytes 2021-03-09T03:00:37.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 520161280 bytes 2021-03-09T03:00:37.666 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 550699008 bytes 2021-03-09T03:00:37.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 569379840 bytes 2021-03-09T03:00:38.111 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 583631872 bytes 2021-03-09T03:00:38.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 633669632 bytes 2021-03-09T03:00:38.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 654830592 bytes 2021-03-09T03:00:38.877 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 686413824 bytes 2021-03-09T03:00:39.202 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 750204928 bytes 2021-03-09T03:00:39.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 768632832 bytes 2021-03-09T03:00:39.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 815785984 bytes 2021-03-09T03:00:39.990 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 864771072 bytes 2021-03-09T03:00:40.233 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 898130944 bytes 2021-03-09T03:00:40.476 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 933095424 bytes 2021-03-09T03:00:40.666 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 938068992 bytes 2021-03-09T03:00:41.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1021971456 bytes 2021-03-09T03:00:41.305 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1051619328 bytes 2021-03-09T03:00:41.520 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1074644992 bytes 2021-03-09T03:00:41.842 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1136918528 bytes 2021-03-09T03:00:42.065 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1169603584 bytes 2021-03-09T03:00:42.462 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1251223552 bytes 2021-03-09T03:00:42.820 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1313339392 bytes 2021-03-09T03:00:43.092 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1348215808 bytes 2021-03-09T03:00:43.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1407196160 bytes 2021-03-09T03:00:43.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1450129408 bytes 2021-03-09T03:00:43.995 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1505703936 bytes 2021-03-09T03:00:44.200 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1528293376 bytes 2021-03-09T03:00:44.467 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4000 IOs, 1557967872 bytes 2021-03-09T03:00:44.531 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:00:44.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:00:44.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:00:44.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:00:44.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_2 2021-03-09T03:00:44.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_2 snapshot 128M 2021-03-09T03:00:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:44.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:44.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_2 2021-03-09T03:00:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:00:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:00:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:44.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_2 128M 2021-03-09T03:00:44.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:44.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:44.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:44.537 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:44.538 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_2 2021-03-09T03:00:44.538 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:44.538 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:00:44.538 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:00:44.538 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:00:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:44.539 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_2 2021-03-09T03:00:44.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:00:44.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_2 snapshot 2021-03-09T03:00:44.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:44.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:44.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_2 2021-03-09T03:00:44.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:00:44.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_2 snapshot 2021-03-09T03:00:45.480 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:00:45.497 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:00:45.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_2 2021-03-09T03:00:45.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:45.498 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:45.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_2 2021-03-09T03:00:45.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:00:45.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=8 2021-03-09T03:00:45.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:00:45.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_2 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:00:45.795 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 18902016 bytes 2021-03-09T03:00:46.011 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 28665856 bytes 2021-03-09T03:00:46.267 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 63352832 bytes 2021-03-09T03:00:46.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 70367232 bytes 2021-03-09T03:00:46.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 109523968 bytes 2021-03-09T03:00:46.952 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 139536384 bytes 2021-03-09T03:00:47.150 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 144882688 bytes 2021-03-09T03:00:47.370 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 185495552 bytes 2021-03-09T03:00:47.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 200913920 bytes 2021-03-09T03:00:48.021 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 321035264 bytes 2021-03-09T03:00:48.524 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 429242368 bytes 2021-03-09T03:00:48.838 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 473740288 bytes 2021-03-09T03:00:49.145 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 519629824 bytes 2021-03-09T03:00:49.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 547734528 bytes 2021-03-09T03:00:49.718 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 589961216 bytes 2021-03-09T03:00:50.136 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 681103360 bytes 2021-03-09T03:00:50.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 708039680 bytes 2021-03-09T03:00:50.674 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 738839552 bytes 2021-03-09T03:00:50.938 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 788805632 bytes 2021-03-09T03:00:51.314 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 835473408 bytes 2021-03-09T03:00:51.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 864574464 bytes 2021-03-09T03:00:51.865 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 905841664 bytes 2021-03-09T03:00:52.121 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 929937408 bytes 2021-03-09T03:00:52.381 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 955510784 bytes 2021-03-09T03:00:52.745 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 983782400 bytes 2021-03-09T03:00:53.027 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1019624448 bytes 2021-03-09T03:00:53.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1090219008 bytes 2021-03-09T03:00:53.507 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:00:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:00:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:00:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:00:53.508 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_3 2021-03-09T03:00:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_3 snapshot 128M 2021-03-09T03:00:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:53.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_3 2021-03-09T03:00:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:00:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:00:53.510 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_3 128M 2021-03-09T03:00:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:53.511 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_3 2021-03-09T03:00:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:53.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:00:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:00:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:00:53.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:00:53.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_3 2021-03-09T03:00:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:00:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_3 snapshot 2021-03-09T03:00:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:53.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:53.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_3 2021-03-09T03:00:53.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:00:53.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_3 snapshot 2021-03-09T03:00:54.520 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:00:54.542 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:00:54.542 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_3 2021-03-09T03:00:54.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:00:54.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:00:54.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_3 2021-03-09T03:00:54.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:00:54.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=11 2021-03-09T03:00:54.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:00:54.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_3 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:00:54.819 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 7366656 bytes 2021-03-09T03:00:55.048 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 38976512 bytes 2021-03-09T03:00:55.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 110522368 bytes 2021-03-09T03:00:55.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 113288192 bytes 2021-03-09T03:00:55.918 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 199774208 bytes 2021-03-09T03:00:56.154 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 241033216 bytes 2021-03-09T03:00:56.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 304741376 bytes 2021-03-09T03:00:56.788 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 342227968 bytes 2021-03-09T03:00:57.067 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 385870848 bytes 2021-03-09T03:00:57.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 441204736 bytes 2021-03-09T03:00:57.718 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 501568512 bytes 2021-03-09T03:00:58.012 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 541058048 bytes 2021-03-09T03:00:58.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 652997632 bytes 2021-03-09T03:00:58.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 656561152 bytes 2021-03-09T03:00:59.022 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 745188352 bytes 2021-03-09T03:00:59.313 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 793782272 bytes 2021-03-09T03:00:59.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 831831040 bytes 2021-03-09T03:00:59.858 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 863129600 bytes 2021-03-09T03:01:00.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 863734784 bytes 2021-03-09T03:01:00.311 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 882322432 bytes 2021-03-09T03:01:00.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 905219072 bytes 2021-03-09T03:01:00.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 951569408 bytes 2021-03-09T03:01:01.100 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 974448640 bytes 2021-03-09T03:01:01.468 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1051283456 bytes 2021-03-09T03:01:01.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1078990848 bytes 2021-03-09T03:01:01.924 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:01:01.920+0000 7f082a074700 -1 reset not still connected to 0x55a97b99f380 2021-03-09T03:01:01.933 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1079617536 bytes 2021-03-09T03:01:02.218 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1119973376 bytes 2021-03-09T03:01:02.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1169217536 bytes 2021-03-09T03:01:02.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1197171712 bytes 2021-03-09T03:01:03.120 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1206716416 bytes 2021-03-09T03:01:03.473 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1224189952 bytes 2021-03-09T03:01:03.751 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1248700416 bytes 2021-03-09T03:01:04.089 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1314223104 bytes 2021-03-09T03:01:04.433 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1363624960 bytes 2021-03-09T03:01:04.780 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1420282880 bytes 2021-03-09T03:01:05.033 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1433154560 bytes 2021-03-09T03:01:05.310 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1462279168 bytes 2021-03-09T03:01:05.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:01:05.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:01:05.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:01:05.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:01:05.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_4 2021-03-09T03:01:05.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_4 snapshot 128M 2021-03-09T03:01:05.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:05.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:05.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:05.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_4 2021-03-09T03:01:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:01:05.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_4 128M 2021-03-09T03:01:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:05.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:05.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_4 2021-03-09T03:01:05.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:05.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:01:05.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:01:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:01:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:05.554 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_4 2021-03-09T03:01:05.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:01:05.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_4 snapshot 2021-03-09T03:01:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_4 2021-03-09T03:01:05.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:05.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_4 snapshot 2021-03-09T03:01:05.823 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:01:05.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:01:05.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_4 2021-03-09T03:01:05.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:05.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:05.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_4 2021-03-09T03:01:05.846 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:01:05.847 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=10 2021-03-09T03:01:05.847 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:01:05.848 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_4 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:01:06.227 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 57888768 bytes 2021-03-09T03:01:06.472 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 83432448 bytes 2021-03-09T03:01:06.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 142980096 bytes 2021-03-09T03:01:07.029 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 181064704 bytes 2021-03-09T03:01:07.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 233061376 bytes 2021-03-09T03:01:07.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 274079744 bytes 2021-03-09T03:01:08.148 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 349454336 bytes 2021-03-09T03:01:08.453 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 374755328 bytes 2021-03-09T03:01:08.742 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 398609408 bytes 2021-03-09T03:01:09.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 436105216 bytes 2021-03-09T03:01:09.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 495732736 bytes 2021-03-09T03:01:09.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 535459840 bytes 2021-03-09T03:01:09.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 582280192 bytes 2021-03-09T03:01:10.281 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 632329216 bytes 2021-03-09T03:01:10.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 678221824 bytes 2021-03-09T03:01:10.980 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 745373696 bytes 2021-03-09T03:01:11.240 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 755599360 bytes 2021-03-09T03:01:11.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 808434688 bytes 2021-03-09T03:01:11.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 809058304 bytes 2021-03-09T03:01:12.010 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 825689088 bytes 2021-03-09T03:01:12.296 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 863902720 bytes 2021-03-09T03:01:12.646 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 929089536 bytes 2021-03-09T03:01:13.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 994015232 bytes 2021-03-09T03:01:13.276 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 994634752 bytes 2021-03-09T03:01:13.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1018133504 bytes 2021-03-09T03:01:13.993 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1070324736 bytes 2021-03-09T03:01:14.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1110871040 bytes 2021-03-09T03:01:14.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1166101504 bytes 2021-03-09T03:01:14.932 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1187986432 bytes 2021-03-09T03:01:15.297 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1246852096 bytes 2021-03-09T03:01:15.556 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1269444608 bytes 2021-03-09T03:01:15.851 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:01:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:01:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:01:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:01:15.852 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_5 2021-03-09T03:01:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_5 snapshot 128M 2021-03-09T03:01:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:15.853 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:15.854 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:15.854 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_5 2021-03-09T03:01:15.854 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:15.854 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:15.854 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:01:15.855 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:15.855 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_5 128M 2021-03-09T03:01:15.855 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:15.855 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:15.856 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:15.856 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:15.856 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_5 2021-03-09T03:01:15.856 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:15.856 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:01:15.857 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:01:15.857 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:01:15.857 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:15.857 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_5 2021-03-09T03:01:16.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:01:16.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_5 snapshot 2021-03-09T03:01:16.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:16.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:16.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_5 2021-03-09T03:01:16.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:16.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_5 snapshot 2021-03-09T03:01:17.519 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:01:17.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:01:17.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_5 2021-03-09T03:01:17.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:17.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:17.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_5 2021-03-09T03:01:17.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:01:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=14 2021-03-09T03:01:17.838 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:01:17.839 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_5 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:01:18.210 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 30483456 bytes 2021-03-09T03:01:18.441 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 46158848 bytes 2021-03-09T03:01:18.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 101597184 bytes 2021-03-09T03:01:18.947 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 103469056 bytes 2021-03-09T03:01:19.297 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 166747136 bytes 2021-03-09T03:01:19.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 303149056 bytes 2021-03-09T03:01:19.982 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 306130944 bytes 2021-03-09T03:01:20.309 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 319009792 bytes 2021-03-09T03:01:20.547 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 331874304 bytes 2021-03-09T03:01:21.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 405707776 bytes 2021-03-09T03:01:21.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 464845824 bytes 2021-03-09T03:01:22.015 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 491947008 bytes 2021-03-09T03:01:22.350 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 538177536 bytes 2021-03-09T03:01:22.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 577702912 bytes 2021-03-09T03:01:23.143 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 613562368 bytes 2021-03-09T03:01:23.442 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 643785728 bytes 2021-03-09T03:01:23.874 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 734259200 bytes 2021-03-09T03:01:24.205 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 768120832 bytes 2021-03-09T03:01:24.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 833253376 bytes 2021-03-09T03:01:24.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 854596608 bytes 2021-03-09T03:01:25.236 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 921676800 bytes 2021-03-09T03:01:25.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 999780352 bytes 2021-03-09T03:01:26.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 1075173376 bytes 2021-03-09T03:01:26.377 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1083400192 bytes 2021-03-09T03:01:26.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1098982400 bytes 2021-03-09T03:01:26.940 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1131165696 bytes 2021-03-09T03:01:27.178 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1139768320 bytes 2021-03-09T03:01:27.248 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T03:01:27.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T03:01:27.249 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T03:01:27.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T03:01:27.243+0000 7f502dadc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 22167) UID: 1000 2021-03-09T03:01:27.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:01:27.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1140391936 bytes 2021-03-09T03:01:27.940 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1200748544 bytes 2021-03-09T03:01:28.424 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1290510336 bytes 2021-03-09T03:01:28.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1328608256 bytes 2021-03-09T03:01:29.082 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1354976256 bytes 2021-03-09T03:01:29.250 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T03:01:29.251 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:01:29.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:01:29.251 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T03:01:29.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T03:01:29.252 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T03:01:29.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T03:01:29.257 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T03:01:29.452 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1413441536 bytes 2021-03-09T03:01:29.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1507757056 bytes 2021-03-09T03:01:30.215 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1543242752 bytes 2021-03-09T03:01:30.668 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1622066176 bytes 2021-03-09T03:01:31.119 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1703724032 bytes 2021-03-09T03:01:31.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1758927872 bytes 2021-03-09T03:01:31.757 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1759547392 bytes 2021-03-09T03:01:31.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:01:31.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:01:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:01:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:01:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_6 2021-03-09T03:01:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_6 snapshot 128M 2021-03-09T03:01:31.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:31.846 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_6 2021-03-09T03:01:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:01:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:31.847 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_6 128M 2021-03-09T03:01:31.848 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:31.848 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:31.848 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:31.848 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_6 2021-03-09T03:01:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:01:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:01:31.849 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:01:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:31.850 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_6 2021-03-09T03:01:31.915 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:01:31.916 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_6 snapshot 2021-03-09T03:01:31.916 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:31.916 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:31.916 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_6 2021-03-09T03:01:31.916 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:31.917 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_6 snapshot 2021-03-09T03:01:32.519 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:01:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:01:32.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_6 2021-03-09T03:01:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_6 2021-03-09T03:01:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:01:32.671 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=6 2021-03-09T03:01:32.671 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:01:32.671 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_6 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:01:33.081 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 20733952 bytes 2021-03-09T03:01:33.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 95366144 bytes 2021-03-09T03:01:33.704 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 124058624 bytes 2021-03-09T03:01:33.961 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 157271040 bytes 2021-03-09T03:01:34.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 190667776 bytes 2021-03-09T03:01:34.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 211763200 bytes 2021-03-09T03:01:34.902 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 278723584 bytes 2021-03-09T03:01:35.304 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 364752896 bytes 2021-03-09T03:01:35.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 434816000 bytes 2021-03-09T03:01:36.070 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 515695616 bytes 2021-03-09T03:01:36.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 550571008 bytes 2021-03-09T03:01:36.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 627968000 bytes 2021-03-09T03:01:37.121 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 707261440 bytes 2021-03-09T03:01:37.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 724108288 bytes 2021-03-09T03:01:37.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 732488704 bytes 2021-03-09T03:01:37.945 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 749896704 bytes 2021-03-09T03:01:38.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 814184448 bytes 2021-03-09T03:01:38.676 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:01:38.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:01:38.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:01:38.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:01:38.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_7 2021-03-09T03:01:38.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_7 snapshot 128M 2021-03-09T03:01:38.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:38.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:38.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:38.679 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:38.679 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_7 2021-03-09T03:01:38.679 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:38.679 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:38.679 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:01:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_7 128M 2021-03-09T03:01:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:38.680 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:38.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:38.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:38.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_7 2021-03-09T03:01:38.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:01:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:01:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:01:38.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:38.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_7 2021-03-09T03:01:38.843 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:01:38.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_7 snapshot 2021-03-09T03:01:38.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:38.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:38.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_7 2021-03-09T03:01:38.844 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:38.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_7 snapshot 2021-03-09T03:01:39.556 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:01:39.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:01:39.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_7 2021-03-09T03:01:39.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:39.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:39.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_7 2021-03-09T03:01:39.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:01:39.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=10 2021-03-09T03:01:39.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:01:39.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_7 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:01:39.975 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 12431360 bytes 2021-03-09T03:01:40.234 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 21797888 bytes 2021-03-09T03:01:40.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 60360704 bytes 2021-03-09T03:01:40.825 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 103246848 bytes 2021-03-09T03:01:41.252 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 227601408 bytes 2021-03-09T03:01:41.626 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 285123584 bytes 2021-03-09T03:01:41.959 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 340301824 bytes 2021-03-09T03:01:42.230 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 359799808 bytes 2021-03-09T03:01:42.534 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 399718400 bytes 2021-03-09T03:01:42.797 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 400323584 bytes 2021-03-09T03:01:43.062 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 424439808 bytes 2021-03-09T03:01:43.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 457408512 bytes 2021-03-09T03:01:43.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 494415872 bytes 2021-03-09T03:01:44.015 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 561758208 bytes 2021-03-09T03:01:44.410 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 633563136 bytes 2021-03-09T03:01:44.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 655111168 bytes 2021-03-09T03:01:44.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 655734784 bytes 2021-03-09T03:01:45.139 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 656356352 bytes 2021-03-09T03:01:45.539 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 731492352 bytes 2021-03-09T03:01:45.810 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 754709504 bytes 2021-03-09T03:01:46.036 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 763794432 bytes 2021-03-09T03:01:46.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 863144960 bytes 2021-03-09T03:01:46.812 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 924427264 bytes 2021-03-09T03:01:47.098 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 939942912 bytes 2021-03-09T03:01:47.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 965402624 bytes 2021-03-09T03:01:47.633 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 966016000 bytes 2021-03-09T03:01:47.864 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 971252736 bytes 2021-03-09T03:01:48.054 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4927730853066238) > weight (0.25) 2021-03-09T03:01:48.055 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T03:01:48.055 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T03:01:48.056 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T03:01:48.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T03:01:48.051+0000 7f719344c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 23322) UID: 1000 2021-03-09T03:01:48.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1019279360 bytes 2021-03-09T03:01:48.664 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:01:48.722 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1094528000 bytes 2021-03-09T03:01:49.004 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1118930944 bytes 2021-03-09T03:01:49.383 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1184389120 bytes 2021-03-09T03:01:49.671 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:01:49.672 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:01:49.672 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:01:49.673 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:01:49.673 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_8 2021-03-09T03:01:49.673 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_8 snapshot 128M 2021-03-09T03:01:49.673 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:49.674 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_8 2021-03-09T03:01:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:01:49.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:49.676 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_8 128M 2021-03-09T03:01:49.676 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:49.676 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:49.676 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:49.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:49.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_8 2021-03-09T03:01:49.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:49.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:01:49.677 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:01:49.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:01:49.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:49.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_8 2021-03-09T03:01:49.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:01:49.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_8 snapshot 2021-03-09T03:01:49.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:49.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:49.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_8 2021-03-09T03:01:49.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:49.830 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_8 snapshot 2021-03-09T03:01:50.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:01:50.521 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:01:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_8 2021-03-09T03:01:50.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_8 2021-03-09T03:01:50.523 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:01:50.524 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=6 2021-03-09T03:01:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:01:50.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_8 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:01:51.024 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 101108736 bytes 2021-03-09T03:01:51.370 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 147204096 bytes 2021-03-09T03:01:51.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 200871936 bytes 2021-03-09T03:01:51.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 230631424 bytes 2021-03-09T03:01:52.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 324046848 bytes 2021-03-09T03:01:52.678 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 324645888 bytes 2021-03-09T03:01:52.878 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 325270528 bytes 2021-03-09T03:01:53.057 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T03:01:53.057 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:01:53.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:01:53.059 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T03:01:53.059 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T03:01:53.060 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T03:01:53.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T03:01:53.063 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T03:01:53.117 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 328966144 bytes 2021-03-09T03:01:53.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 376133632 bytes 2021-03-09T03:01:54.037 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 461633536 bytes 2021-03-09T03:01:54.318 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 483557376 bytes 2021-03-09T03:01:54.742 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 581228544 bytes 2021-03-09T03:01:55.050 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 609120256 bytes 2021-03-09T03:01:55.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 648003584 bytes 2021-03-09T03:01:55.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 683431936 bytes 2021-03-09T03:01:55.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 711427072 bytes 2021-03-09T03:01:56.377 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 784218112 bytes 2021-03-09T03:01:56.527 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:01:56.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:01:56.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:01:56.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:01:56.528 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_9 2021-03-09T03:01:56.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_9 snapshot 128M 2021-03-09T03:01:56.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:56.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:56.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:56.529 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_9 2021-03-09T03:01:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:56.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:01:56.531 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:56.531 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_9 128M 2021-03-09T03:01:56.531 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:56.531 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:56.531 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:56.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:56.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_9 2021-03-09T03:01:56.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:56.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:01:56.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:01:56.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:01:56.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:01:56.533 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_9 2021-03-09T03:01:56.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:01:56.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_9 snapshot 2021-03-09T03:01:56.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:56.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:56.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_9 2021-03-09T03:01:56.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:01:56.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_9 snapshot 2021-03-09T03:01:57.541 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:01:57.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:01:57.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_9 2021-03-09T03:01:57.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:01:57.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:01:57.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_9 2021-03-09T03:01:57.563 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:01:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=8 2021-03-09T03:01:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:01:57.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_9 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:01:57.902 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 21340160 bytes 2021-03-09T03:01:58.141 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 39117824 bytes 2021-03-09T03:01:58.474 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 80400384 bytes 2021-03-09T03:01:58.818 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 140401664 bytes 2021-03-09T03:01:59.143 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 191078400 bytes 2021-03-09T03:01:59.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 241099776 bytes 2021-03-09T03:01:59.774 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 260425728 bytes 2021-03-09T03:02:00.082 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 283107328 bytes 2021-03-09T03:02:00.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 318025728 bytes 2021-03-09T03:02:00.637 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 326424576 bytes 2021-03-09T03:02:00.897 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 336758784 bytes 2021-03-09T03:02:01.264 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 393645056 bytes 2021-03-09T03:02:01.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 413944832 bytes 2021-03-09T03:02:01.924 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 465988608 bytes 2021-03-09T03:02:02.382 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 570838016 bytes 2021-03-09T03:02:02.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 591444992 bytes 2021-03-09T03:02:03.009 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 634787840 bytes 2021-03-09T03:02:03.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 639868928 bytes 2021-03-09T03:02:03.889 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 692292608 bytes 2021-03-09T03:02:04.165 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 711757824 bytes 2021-03-09T03:02:04.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 747790336 bytes 2021-03-09T03:02:04.776 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 778668032 bytes 2021-03-09T03:02:05.168 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 825067520 bytes 2021-03-09T03:02:05.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 825667584 bytes 2021-03-09T03:02:05.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:02:05.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:02:05.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:02:05.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:02:05.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_10 2021-03-09T03:02:05.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_10 snapshot 128M 2021-03-09T03:02:05.572 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:05.572 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:05.572 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:05.572 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_10 2021-03-09T03:02:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:02:05.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:05.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_10 128M 2021-03-09T03:02:05.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:05.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:05.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:05.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:05.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_10 2021-03-09T03:02:05.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:05.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:02:05.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:02:05.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:02:05.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:05.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_10 2021-03-09T03:02:06.093 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:02:06.093 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_10 snapshot 2021-03-09T03:02:06.093 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:06.094 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:06.094 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_10 2021-03-09T03:02:06.094 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:06.094 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_10 snapshot 2021-03-09T03:02:06.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:02:06.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:02:06.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_10 2021-03-09T03:02:06.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:06.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:06.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_10 2021-03-09T03:02:06.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:02:06.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=6 2021-03-09T03:02:06.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:02:06.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_10 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:02:06.980 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 32414720 bytes 2021-03-09T03:02:07.264 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 53878784 bytes 2021-03-09T03:02:07.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 91833344 bytes 2021-03-09T03:02:07.929 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 131600384 bytes 2021-03-09T03:02:08.244 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 157617152 bytes 2021-03-09T03:02:08.645 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 222609408 bytes 2021-03-09T03:02:09.052 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 314365952 bytes 2021-03-09T03:02:09.333 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 321547264 bytes 2021-03-09T03:02:09.702 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 382301184 bytes 2021-03-09T03:02:09.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 382943232 bytes 2021-03-09T03:02:10.288 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 448091136 bytes 2021-03-09T03:02:10.532 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 470264832 bytes 2021-03-09T03:02:10.874 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 517672960 bytes 2021-03-09T03:02:11.293 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 602015744 bytes 2021-03-09T03:02:11.558 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 613945344 bytes 2021-03-09T03:02:11.928 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 658152448 bytes 2021-03-09T03:02:12.230 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 673963008 bytes 2021-03-09T03:02:12.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 700365824 bytes 2021-03-09T03:02:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:02:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:02:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:02:12.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:02:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_11 2021-03-09T03:02:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_11 snapshot 128M 2021-03-09T03:02:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:12.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_11 2021-03-09T03:02:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:12.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:02:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_11 128M 2021-03-09T03:02:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:12.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_11 2021-03-09T03:02:12.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:12.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:02:12.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:02:12.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:02:12.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:12.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_11 2021-03-09T03:02:12.825 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:02:12.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_11 snapshot 2021-03-09T03:02:12.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:12.826 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:12.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_11 2021-03-09T03:02:12.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:12.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_11 snapshot 2021-03-09T03:02:13.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:02:13.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:02:13.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_11 2021-03-09T03:02:13.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:13.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:13.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_11 2021-03-09T03:02:13.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:02:13.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=9 2021-03-09T03:02:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:02:13.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_11 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:02:13.970 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 35406848 bytes 2021-03-09T03:02:14.272 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 62783488 bytes 2021-03-09T03:02:14.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 137048064 bytes 2021-03-09T03:02:15.111 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 212097024 bytes 2021-03-09T03:02:15.420 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 243612672 bytes 2021-03-09T03:02:15.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 284236800 bytes 2021-03-09T03:02:16.118 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 331569152 bytes 2021-03-09T03:02:16.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 342473728 bytes 2021-03-09T03:02:16.720 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 380875776 bytes 2021-03-09T03:02:17.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 402294784 bytes 2021-03-09T03:02:17.439 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 463958016 bytes 2021-03-09T03:02:17.763 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 475812864 bytes 2021-03-09T03:02:18.180 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 499296256 bytes 2021-03-09T03:02:18.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 536626176 bytes 2021-03-09T03:02:18.745 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 565398528 bytes 2021-03-09T03:02:19.068 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 592019456 bytes 2021-03-09T03:02:19.426 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 624260096 bytes 2021-03-09T03:02:19.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 644036608 bytes 2021-03-09T03:02:20.024 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 675204096 bytes 2021-03-09T03:02:20.428 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 732514304 bytes 2021-03-09T03:02:20.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 775356416 bytes 2021-03-09T03:02:21.107 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 804711424 bytes 2021-03-09T03:02:21.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 914740224 bytes 2021-03-09T03:02:21.973 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 966936576 bytes 2021-03-09T03:02:22.403 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1070843904 bytes 2021-03-09T03:02:22.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:02:22.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:02:22.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:02:22.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:02:22.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_12 2021-03-09T03:02:22.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_12 snapshot 128M 2021-03-09T03:02:22.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:22.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:22.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:22.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:22.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_12 2021-03-09T03:02:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:22.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:02:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:22.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_12 128M 2021-03-09T03:02:22.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:22.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:22.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:22.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:22.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_12 2021-03-09T03:02:22.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:22.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:02:22.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:02:22.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:02:22.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:22.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_12 2021-03-09T03:02:22.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:02:22.892 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_12 snapshot 2021-03-09T03:02:22.892 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:22.892 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:22.893 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_12 2021-03-09T03:02:22.893 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:22.893 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_12 snapshot 2021-03-09T03:02:23.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:02:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:02:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_12 2021-03-09T03:02:23.540 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_12 2021-03-09T03:02:23.541 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:02:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=8 2021-03-09T03:02:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:02:23.542 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_12 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:02:24.048 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 96343040 bytes 2021-03-09T03:02:24.407 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 143287296 bytes 2021-03-09T03:02:24.719 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 180873216 bytes 2021-03-09T03:02:24.986 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 181478400 bytes 2021-03-09T03:02:25.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 224141312 bytes 2021-03-09T03:02:25.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 292631552 bytes 2021-03-09T03:02:26.107 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 342579200 bytes 2021-03-09T03:02:26.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 399292416 bytes 2021-03-09T03:02:26.938 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 464947200 bytes 2021-03-09T03:02:27.282 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 497822720 bytes 2021-03-09T03:02:27.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 534234112 bytes 2021-03-09T03:02:27.919 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 552227840 bytes 2021-03-09T03:02:28.230 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 582904832 bytes 2021-03-09T03:02:28.764 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 650390528 bytes 2021-03-09T03:02:29.261 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 734051328 bytes 2021-03-09T03:02:29.625 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 783326208 bytes 2021-03-09T03:02:30.002 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 844009472 bytes 2021-03-09T03:02:30.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 878661632 bytes 2021-03-09T03:02:30.786 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 961182720 bytes 2021-03-09T03:02:31.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 961816576 bytes 2021-03-09T03:02:31.307 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 969526272 bytes 2021-03-09T03:02:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:02:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:02:31.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:02:31.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:02:31.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_13 2021-03-09T03:02:31.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_13 snapshot 128M 2021-03-09T03:02:31.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:31.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:31.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:31.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:31.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_13 2021-03-09T03:02:31.550 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:31.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:31.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:02:31.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:31.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_13 128M 2021-03-09T03:02:31.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:31.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:31.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:31.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:31.552 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_13 2021-03-09T03:02:31.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:31.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:02:31.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:02:31.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:02:31.554 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:31.554 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_13 2021-03-09T03:02:31.909 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:02:31.909 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_13 snapshot 2021-03-09T03:02:31.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:31.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:31.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_13 2021-03-09T03:02:31.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:31.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_13 snapshot 2021-03-09T03:02:32.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:02:32.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:02:32.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_13 2021-03-09T03:02:32.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:32.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:32.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_13 2021-03-09T03:02:32.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:02:32.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=14 2021-03-09T03:02:32.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:02:32.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_13 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:02:32.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 9327616 bytes 2021-03-09T03:02:33.211 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 48807936 bytes 2021-03-09T03:02:33.491 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 75952128 bytes 2021-03-09T03:02:33.903 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 155071488 bytes 2021-03-09T03:02:34.282 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 211520512 bytes 2021-03-09T03:02:34.716 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 293226496 bytes 2021-03-09T03:02:35.039 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 321242112 bytes 2021-03-09T03:02:35.339 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 328084480 bytes 2021-03-09T03:02:35.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 378074112 bytes 2021-03-09T03:02:36.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 453494784 bytes 2021-03-09T03:02:36.444 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 502767616 bytes 2021-03-09T03:02:36.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 542646272 bytes 2021-03-09T03:02:37.118 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 560297984 bytes 2021-03-09T03:02:37.555 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 642807808 bytes 2021-03-09T03:02:37.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 688517120 bytes 2021-03-09T03:02:38.234 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 729515008 bytes 2021-03-09T03:02:38.702 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 773399552 bytes 2021-03-09T03:02:39.242 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 885289984 bytes 2021-03-09T03:02:39.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 910566400 bytes 2021-03-09T03:02:39.828 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 928834560 bytes 2021-03-09T03:02:40.178 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 978293760 bytes 2021-03-09T03:02:40.499 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 1009451008 bytes 2021-03-09T03:02:40.893 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 1072588800 bytes 2021-03-09T03:02:41.263 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1116102656 bytes 2021-03-09T03:02:41.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1162961920 bytes 2021-03-09T03:02:41.913 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1194251264 bytes 2021-03-09T03:02:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1260470272 bytes 2021-03-09T03:02:42.757 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1312423936 bytes 2021-03-09T03:02:43.132 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1350648832 bytes 2021-03-09T03:02:43.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1435322368 bytes 2021-03-09T03:02:44.248 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1500682240 bytes 2021-03-09T03:02:44.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1546323968 bytes 2021-03-09T03:02:44.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1549978624 bytes 2021-03-09T03:02:45.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1586226176 bytes 2021-03-09T03:02:45.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1629624320 bytes 2021-03-09T03:02:45.854 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1635234816 bytes 2021-03-09T03:02:46.262 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1692362752 bytes 2021-03-09T03:02:46.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:02:46.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:02:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:02:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:02:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_14 2021-03-09T03:02:46.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_14 snapshot 128M 2021-03-09T03:02:46.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:46.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:46.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:46.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:46.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_14 2021-03-09T03:02:46.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:46.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:46.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:02:46.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:46.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_14 128M 2021-03-09T03:02:46.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:46.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:46.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:46.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:46.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_14 2021-03-09T03:02:46.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:46.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:02:46.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:02:46.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:02:46.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:46.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_14 2021-03-09T03:02:46.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:02:46.652 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_14 snapshot 2021-03-09T03:02:46.652 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:46.652 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:46.652 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_14 2021-03-09T03:02:46.653 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:46.653 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_14 snapshot 2021-03-09T03:02:47.566 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:02:47.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:02:47.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_14 2021-03-09T03:02:47.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:47.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:47.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_14 2021-03-09T03:02:47.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:02:47.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=10 2021-03-09T03:02:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:02:47.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_14 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:02:47.982 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 38378496 bytes 2021-03-09T03:02:48.338 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 89775104 bytes 2021-03-09T03:02:48.720 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 150752256 bytes 2021-03-09T03:02:48.958 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 162718720 bytes 2021-03-09T03:02:49.431 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 240294912 bytes 2021-03-09T03:02:49.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 252872704 bytes 2021-03-09T03:02:50.306 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 312700928 bytes 2021-03-09T03:02:50.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 358939648 bytes 2021-03-09T03:02:50.982 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 386628608 bytes 2021-03-09T03:02:51.263 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 390998016 bytes 2021-03-09T03:02:51.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 446757888 bytes 2021-03-09T03:02:52.142 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 554501120 bytes 2021-03-09T03:02:52.556 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 624825344 bytes 2021-03-09T03:02:52.916 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 646034432 bytes 2021-03-09T03:02:53.342 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 715160576 bytes 2021-03-09T03:02:53.704 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 757641216 bytes 2021-03-09T03:02:54.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 789843968 bytes 2021-03-09T03:02:54.839 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 876733440 bytes 2021-03-09T03:02:55.225 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 918694912 bytes 2021-03-09T03:02:55.564 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 939251712 bytes 2021-03-09T03:02:55.853 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 961968128 bytes 2021-03-09T03:02:56.208 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 988746752 bytes 2021-03-09T03:02:56.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 1029160960 bytes 2021-03-09T03:02:56.987 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1085824000 bytes 2021-03-09T03:02:57.379 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1127738368 bytes 2021-03-09T03:02:57.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:02:57.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:02:57.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:02:57.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:02:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_15 2021-03-09T03:02:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_15 snapshot 128M 2021-03-09T03:02:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:57.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:57.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:57.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:57.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_15 2021-03-09T03:02:57.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:57.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:57.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:02:57.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:57.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_15 128M 2021-03-09T03:02:57.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:57.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:57.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:57.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:57.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_15 2021-03-09T03:02:57.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:57.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:02:57.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:02:57.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:02:57.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:02:57.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_15 2021-03-09T03:02:57.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:02:57.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_15 snapshot 2021-03-09T03:02:57.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:57.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:57.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_15 2021-03-09T03:02:57.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:02:57.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_15 snapshot 2021-03-09T03:02:58.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:02:58.574 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:02:58.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_15 2021-03-09T03:02:58.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:02:58.575 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:02:58.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_15 2021-03-09T03:02:58.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:02:58.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=6 2021-03-09T03:02:58.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:02:58.577 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_15 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:02:58.957 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 48697344 bytes 2021-03-09T03:02:59.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 105707520 bytes 2021-03-09T03:02:59.950 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 116162560 bytes 2021-03-09T03:03:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 162518016 bytes 2021-03-09T03:03:00.728 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 216512512 bytes 2021-03-09T03:03:01.045 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 250346496 bytes 2021-03-09T03:03:01.409 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 285704192 bytes 2021-03-09T03:03:01.750 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 308388864 bytes 2021-03-09T03:03:02.160 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 352872448 bytes 2021-03-09T03:03:02.464 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 372593664 bytes 2021-03-09T03:03:02.787 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 391875584 bytes 2021-03-09T03:03:03.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 392496128 bytes 2021-03-09T03:03:03.418 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 415533056 bytes 2021-03-09T03:03:03.750 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 429185024 bytes 2021-03-09T03:03:04.190 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 491107328 bytes 2021-03-09T03:03:04.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:03:04.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:03:04.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:03:04.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:03:04.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_16 2021-03-09T03:03:04.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_16 snapshot 128M 2021-03-09T03:03:04.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:04.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:04.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:04.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:04.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_16 2021-03-09T03:03:04.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:04.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:03:04.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:03:04.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:04.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_16 128M 2021-03-09T03:03:04.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:04.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:04.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_16 2021-03-09T03:03:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:04.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:03:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:03:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:03:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:04.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_16 2021-03-09T03:03:04.665 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:03:04.666 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_16 snapshot 2021-03-09T03:03:04.666 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:04.666 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:04.667 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_16 2021-03-09T03:03:04.667 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:03:04.667 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_16 snapshot 2021-03-09T03:03:05.562 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:03:05.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:03:05.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_16 2021-03-09T03:03:05.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:05.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:05.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_16 2021-03-09T03:03:05.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:03:05.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=13 2021-03-09T03:03:05.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:03:05.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_16 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:03:06.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 62649344 bytes 2021-03-09T03:03:06.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 84436992 bytes 2021-03-09T03:03:06.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 100468736 bytes 2021-03-09T03:03:06.997 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 127460352 bytes 2021-03-09T03:03:07.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 200089600 bytes 2021-03-09T03:03:07.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 260550656 bytes 2021-03-09T03:03:08.113 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 303480832 bytes 2021-03-09T03:03:08.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 378925056 bytes 2021-03-09T03:03:08.933 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 441292800 bytes 2021-03-09T03:03:09.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 506752000 bytes 2021-03-09T03:03:09.741 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 556563456 bytes 2021-03-09T03:03:10.096 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 575565824 bytes 2021-03-09T03:03:10.444 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 611517440 bytes 2021-03-09T03:03:10.790 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 622844928 bytes 2021-03-09T03:03:11.074 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 653479936 bytes 2021-03-09T03:03:11.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 675430400 bytes 2021-03-09T03:03:11.777 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 723634176 bytes 2021-03-09T03:03:12.198 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 780950528 bytes 2021-03-09T03:03:12.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 859792384 bytes 2021-03-09T03:03:13.037 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 923076608 bytes 2021-03-09T03:03:13.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 951518208 bytes 2021-03-09T03:03:13.736 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 979740672 bytes 2021-03-09T03:03:14.089 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 999418880 bytes 2021-03-09T03:03:14.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1058242560 bytes 2021-03-09T03:03:15.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1143141376 bytes 2021-03-09T03:03:15.436 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1161779200 bytes 2021-03-09T03:03:15.816 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1197566976 bytes 2021-03-09T03:03:16.205 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1252076544 bytes 2021-03-09T03:03:16.258 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.43672895872219) > weight (0.25) 2021-03-09T03:03:16.259 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T03:03:16.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T03:03:16.260 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T03:03:16.372 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T03:03:16.361+0000 7ff0125b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 20050) UID: 1000 2021-03-09T03:03:16.561 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1290867712 bytes 2021-03-09T03:03:17.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:03:17.038 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1390960640 bytes 2021-03-09T03:03:17.261 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T03:03:17.261 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:03:17.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:03:17.262 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T03:03:17.262 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T03:03:17.263 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T03:03:17.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T03:03:17.293 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T03:03:17.453 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1447453696 bytes 2021-03-09T03:03:17.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1503974400 bytes 2021-03-09T03:03:18.276 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1523760128 bytes 2021-03-09T03:03:18.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:03:18.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:03:18.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:03:18.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:03:18.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_17 2021-03-09T03:03:18.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_17 snapshot 128M 2021-03-09T03:03:18.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:18.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:18.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:18.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:18.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_17 2021-03-09T03:03:18.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:18.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:03:18.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:03:18.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:18.602 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_17 128M 2021-03-09T03:03:18.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:18.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:18.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:18.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:18.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_17 2021-03-09T03:03:18.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:18.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:03:18.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:03:18.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:03:18.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:18.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_17 2021-03-09T03:03:18.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:03:18.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_17 snapshot 2021-03-09T03:03:18.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:18.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:18.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_17 2021-03-09T03:03:18.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:03:18.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_17 snapshot 2021-03-09T03:03:19.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:03:19.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:03:19.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_17 2021-03-09T03:03:19.587 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:19.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:19.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_17 2021-03-09T03:03:19.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:03:19.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=14 2021-03-09T03:03:19.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:03:19.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_17 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:03:20.067 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 52035584 bytes 2021-03-09T03:03:20.432 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 119521280 bytes 2021-03-09T03:03:20.793 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 143766528 bytes 2021-03-09T03:03:21.164 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 187599872 bytes 2021-03-09T03:03:21.628 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 291606528 bytes 2021-03-09T03:03:21.979 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 326821888 bytes 2021-03-09T03:03:22.271 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 342150144 bytes 2021-03-09T03:03:22.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 428425216 bytes 2021-03-09T03:03:23.117 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 457167872 bytes 2021-03-09T03:03:23.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 536216576 bytes 2021-03-09T03:03:24.137 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 659391488 bytes 2021-03-09T03:03:24.814 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 694634496 bytes 2021-03-09T03:03:25.230 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 748929024 bytes 2021-03-09T03:03:25.553 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 774703104 bytes 2021-03-09T03:03:25.894 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 797535232 bytes 2021-03-09T03:03:26.177 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 817331200 bytes 2021-03-09T03:03:26.590 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 875380736 bytes 2021-03-09T03:03:26.996 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 919202816 bytes 2021-03-09T03:03:27.406 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 981638144 bytes 2021-03-09T03:03:27.835 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 1038228480 bytes 2021-03-09T03:03:28.313 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 1122385920 bytes 2021-03-09T03:03:28.663 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 1156362240 bytes 2021-03-09T03:03:29.064 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 1203441664 bytes 2021-03-09T03:03:29.519 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1268646912 bytes 2021-03-09T03:03:29.806 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1310927872 bytes 2021-03-09T03:03:30.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1375151104 bytes 2021-03-09T03:03:30.645 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1391517696 bytes 2021-03-09T03:03:31.043 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1432545280 bytes 2021-03-09T03:03:31.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1547484160 bytes 2021-03-09T03:03:31.986 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1600519168 bytes 2021-03-09T03:03:32.294 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1610780672 bytes 2021-03-09T03:03:32.663 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1636818944 bytes 2021-03-09T03:03:32.829 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T03:03:32.821+0000 7fc0e0d45700 -1 reset not still connected to 0x55e8d5744750 2021-03-09T03:03:32.998 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1666478080 bytes 2021-03-09T03:03:33.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1734841344 bytes 2021-03-09T03:03:33.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:03:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:03:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:03:33.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:03:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_18 2021-03-09T03:03:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_18 snapshot 128M 2021-03-09T03:03:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:33.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:33.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:33.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_18 2021-03-09T03:03:33.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:33.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:03:33.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:03:33.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:33.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_18 128M 2021-03-09T03:03:33.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:33.597 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:33.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:33.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:33.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_18 2021-03-09T03:03:33.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:33.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:03:33.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:03:33.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:03:33.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:33.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_18 2021-03-09T03:03:33.785 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:03:33.785 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_18 snapshot 2021-03-09T03:03:33.785 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:33.786 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:33.786 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_18 2021-03-09T03:03:33.786 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:03:33.786 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_18 snapshot 2021-03-09T03:03:34.848 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:03:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:03:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_18 2021-03-09T03:03:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:34.881 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_18 2021-03-09T03:03:34.882 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:03:34.883 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=10 2021-03-09T03:03:34.883 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:03:34.883 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_18 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:03:35.239 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 15616000 bytes 2021-03-09T03:03:35.551 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 17339392 bytes 2021-03-09T03:03:35.849 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 38227968 bytes 2021-03-09T03:03:36.257 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 111378432 bytes 2021-03-09T03:03:36.617 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 156101632 bytes 2021-03-09T03:03:37.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 244359168 bytes 2021-03-09T03:03:37.573 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 294979584 bytes 2021-03-09T03:03:38.015 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 357268480 bytes 2021-03-09T03:03:38.472 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 418869248 bytes 2021-03-09T03:03:38.819 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 453322752 bytes 2021-03-09T03:03:39.203 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 497461248 bytes 2021-03-09T03:03:39.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 508703744 bytes 2021-03-09T03:03:40.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 598075392 bytes 2021-03-09T03:03:40.750 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 703283200 bytes 2021-03-09T03:03:41.165 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 751854592 bytes 2021-03-09T03:03:41.519 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 779298816 bytes 2021-03-09T03:03:42.064 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.31146197849505364) > weight (0.25) 2021-03-09T03:03:42.065 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5291499490939671) > weight (0.25) 2021-03-09T03:03:42.065 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8552572718474636) > weight (0.25) 2021-03-09T03:03:42.065 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6559084965453014) > weight (0.25) 2021-03-09T03:03:42.065 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T03:03:42.075 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 905906176 bytes 2021-03-09T03:03:42.488 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 949026816 bytes 2021-03-09T03:03:42.867 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 982166528 bytes 2021-03-09T03:03:43.302 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 1042247680 bytes 2021-03-09T03:03:43.646 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 1061870592 bytes 2021-03-09T03:03:44.082 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 1127981056 bytes 2021-03-09T03:03:44.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 1128589312 bytes 2021-03-09T03:03:44.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:03:44.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:03:44.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:03:44.889 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:03:44.889 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_19 2021-03-09T03:03:44.889 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_19 snapshot 128M 2021-03-09T03:03:44.889 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:44.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:44.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:44.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:44.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_19 2021-03-09T03:03:44.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:44.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:03:44.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:03:44.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:44.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_19 128M 2021-03-09T03:03:44.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:44.892 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:44.892 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:44.892 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:44.892 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_19 2021-03-09T03:03:44.893 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:44.893 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:03:44.893 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:03:44.894 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:03:44.894 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:44.894 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_19 2021-03-09T03:03:45.050 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:03:45.050 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_19 snapshot 2021-03-09T03:03:45.050 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:45.051 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:45.051 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_19 2021-03-09T03:03:45.051 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:03:45.051 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_19 snapshot 2021-03-09T03:03:45.544 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:03:45.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:03:45.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_19 2021-03-09T03:03:45.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:45.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:45.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_19 2021-03-09T03:03:45.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:03:45.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=7 2021-03-09T03:03:45.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:03:45.571 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_19 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:03:46.018 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 621568 bytes 2021-03-09T03:03:46.374 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 29228032 bytes 2021-03-09T03:03:46.741 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 81076224 bytes 2021-03-09T03:03:47.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 86973440 bytes 2021-03-09T03:03:47.491 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 130182144 bytes 2021-03-09T03:03:47.929 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 186950656 bytes 2021-03-09T03:03:48.291 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 223401984 bytes 2021-03-09T03:03:48.799 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 276377600 bytes 2021-03-09T03:03:49.272 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 342162432 bytes 2021-03-09T03:03:49.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 418447360 bytes 2021-03-09T03:03:50.156 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 450588672 bytes 2021-03-09T03:03:50.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 457117696 bytes 2021-03-09T03:03:50.868 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 506235904 bytes 2021-03-09T03:03:51.187 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 526303232 bytes 2021-03-09T03:03:51.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 526914560 bytes 2021-03-09T03:03:51.867 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 553188352 bytes 2021-03-09T03:03:52.226 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 579348480 bytes 2021-03-09T03:03:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:03:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:03:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:03:52.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:03:52.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_20 2021-03-09T03:03:52.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_20 snapshot 128M 2021-03-09T03:03:52.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:52.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:52.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:52.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:52.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_20 2021-03-09T03:03:52.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:52.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:03:52.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:03:52.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:52.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_20 128M 2021-03-09T03:03:52.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:52.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:52.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:52.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:52.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_20 2021-03-09T03:03:52.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:52.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:03:52.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:03:52.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:03:52.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:03:52.585 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_20 2021-03-09T03:03:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:03:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_20 snapshot 2021-03-09T03:03:52.722 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:52.723 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:52.723 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_20 2021-03-09T03:03:52.723 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:03:52.723 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_20 snapshot 2021-03-09T03:03:53.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:03:53.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:03:53.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_20 2021-03-09T03:03:53.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:03:53.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:03:53.637 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_20 2021-03-09T03:03:53.637 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:03:53.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=10 2021-03-09T03:03:53.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:03:53.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_20 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:03:54.168 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 22004736 bytes 2021-03-09T03:03:54.578 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 69982208 bytes 2021-03-09T03:03:54.896 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 91944960 bytes 2021-03-09T03:03:55.275 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 111742976 bytes 2021-03-09T03:03:55.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 129731584 bytes 2021-03-09T03:03:55.937 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 166173696 bytes 2021-03-09T03:03:56.284 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 191085568 bytes 2021-03-09T03:03:56.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 195679232 bytes 2021-03-09T03:03:56.945 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 242961408 bytes 2021-03-09T03:03:57.313 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 243591168 bytes 2021-03-09T03:03:57.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 273957888 bytes 2021-03-09T03:03:58.130 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 337734656 bytes 2021-03-09T03:03:58.506 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 361424896 bytes 2021-03-09T03:03:58.856 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 409478144 bytes 2021-03-09T03:03:59.290 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 449223680 bytes 2021-03-09T03:03:59.794 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 518359040 bytes 2021-03-09T03:04:00.253 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 537293824 bytes 2021-03-09T03:04:00.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 551692288 bytes 2021-03-09T03:04:01.092 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 561404928 bytes 2021-03-09T03:04:01.455 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 563152896 bytes 2021-03-09T03:04:01.836 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 585101312 bytes 2021-03-09T03:04:02.226 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 610676736 bytes 2021-03-09T03:04:02.736 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 661445632 bytes 2021-03-09T03:04:03.283 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 749713408 bytes 2021-03-09T03:04:03.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 788802560 bytes 2021-03-09T03:04:03.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:04:03.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:04:03.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:04:03.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:04:03.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_21 2021-03-09T03:04:03.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_21 snapshot 128M 2021-03-09T03:04:03.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:03.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:03.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:03.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:03.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_21 2021-03-09T03:04:03.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:03.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:04:03.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:04:03.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:03.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_21 128M 2021-03-09T03:04:03.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:03.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:03.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:03.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:03.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_21 2021-03-09T03:04:03.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:03.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:04:03.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:04:03.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:04:03.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:03.773 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_21 2021-03-09T03:04:03.857 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:04:03.857 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_21 snapshot 2021-03-09T03:04:03.857 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:03.858 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:03.858 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_21 2021-03-09T03:04:03.858 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:04:03.858 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_21 snapshot 2021-03-09T03:04:04.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:04:04.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:04:04.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_21 2021-03-09T03:04:04.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:04.637 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:04.637 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_21 2021-03-09T03:04:04.637 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:04:04.638 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=11 2021-03-09T03:04:04.638 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:04:04.638 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_21 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:04:05.100 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 32068608 bytes 2021-03-09T03:04:05.492 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 50231296 bytes 2021-03-09T03:04:05.837 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 74317824 bytes 2021-03-09T03:04:06.254 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 122396672 bytes 2021-03-09T03:04:06.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 130363392 bytes 2021-03-09T03:04:06.936 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 176716800 bytes 2021-03-09T03:04:07.305 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 198766592 bytes 2021-03-09T03:04:07.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 257502208 bytes 2021-03-09T03:04:08.041 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 279780352 bytes 2021-03-09T03:04:08.466 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 320590848 bytes 2021-03-09T03:04:08.823 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 353981440 bytes 2021-03-09T03:04:09.185 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 387031040 bytes 2021-03-09T03:04:09.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 439254016 bytes 2021-03-09T03:04:10.086 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 481115136 bytes 2021-03-09T03:04:10.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 500915200 bytes 2021-03-09T03:04:10.934 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 518924288 bytes 2021-03-09T03:04:11.369 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 593990656 bytes 2021-03-09T03:04:11.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 641423360 bytes 2021-03-09T03:04:12.190 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 682503168 bytes 2021-03-09T03:04:12.658 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 688252928 bytes 2021-03-09T03:04:13.011 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 713380864 bytes 2021-03-09T03:04:13.364 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 734692352 bytes 2021-03-09T03:04:13.736 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 752420864 bytes 2021-03-09T03:04:14.067 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 765303808 bytes 2021-03-09T03:04:14.383 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 776971264 bytes 2021-03-09T03:04:14.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 845298688 bytes 2021-03-09T03:04:15.317 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 868143104 bytes 2021-03-09T03:04:15.639 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 868772864 bytes 2021-03-09T03:04:15.645 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:04:15.646 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:04:15.646 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:04:15.646 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:04:15.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_22 2021-03-09T03:04:15.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_22 snapshot 128M 2021-03-09T03:04:15.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:15.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:15.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:15.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:15.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_22 2021-03-09T03:04:15.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:15.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:04:15.649 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:04:15.649 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:15.649 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_22 128M 2021-03-09T03:04:15.649 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:15.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:15.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:15.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:15.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_22 2021-03-09T03:04:15.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:15.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:04:15.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:04:15.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:04:15.652 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:15.652 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_22 2021-03-09T03:04:15.800 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:04:15.801 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_22 snapshot 2021-03-09T03:04:15.801 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:15.801 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:15.801 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_22 2021-03-09T03:04:15.802 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:04:15.802 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_22 snapshot 2021-03-09T03:04:16.567 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:04:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:04:16.593 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_22 2021-03-09T03:04:16.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:16.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:16.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_22 2021-03-09T03:04:16.594 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:04:16.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=7 2021-03-09T03:04:16.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:04:16.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_22 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:04:17.023 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 39602176 bytes 2021-03-09T03:04:17.391 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 93089792 bytes 2021-03-09T03:04:17.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 137626624 bytes 2021-03-09T03:04:18.222 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 186592256 bytes 2021-03-09T03:04:18.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 242916352 bytes 2021-03-09T03:04:19.005 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 274548736 bytes 2021-03-09T03:04:19.422 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 317740032 bytes 2021-03-09T03:04:19.840 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 360016896 bytes 2021-03-09T03:04:20.176 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 392602624 bytes 2021-03-09T03:04:20.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 406649856 bytes 2021-03-09T03:04:21.022 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 445158400 bytes 2021-03-09T03:04:21.435 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 516948992 bytes 2021-03-09T03:04:21.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 527537152 bytes 2021-03-09T03:04:22.244 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 599837696 bytes 2021-03-09T03:04:22.640 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 651929600 bytes 2021-03-09T03:04:22.991 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 677520384 bytes 2021-03-09T03:04:23.423 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 705925120 bytes 2021-03-09T03:04:23.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:04:23.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:04:23.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:04:23.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:04:23.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_23 2021-03-09T03:04:23.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_23 snapshot 128M 2021-03-09T03:04:23.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:23.605 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:23.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:23.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:23.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_23 2021-03-09T03:04:23.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:23.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:04:23.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:04:23.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:23.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_23 128M 2021-03-09T03:04:23.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:23.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:23.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:23.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_23 2021-03-09T03:04:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:04:23.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:04:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:04:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:23.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_23 2021-03-09T03:04:23.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:04:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_23 snapshot 2021-03-09T03:04:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_23 2021-03-09T03:04:23.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:04:23.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_23 snapshot 2021-03-09T03:04:24.568 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:04:24.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:04:24.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_23 2021-03-09T03:04:24.599 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:24.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:24.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_23 2021-03-09T03:04:24.600 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:04:24.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=11 2021-03-09T03:04:24.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:04:24.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_23 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:04:25.073 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 55248896 bytes 2021-03-09T03:04:25.433 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 80573440 bytes 2021-03-09T03:04:25.831 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 115464192 bytes 2021-03-09T03:04:26.288 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 201292800 bytes 2021-03-09T03:04:26.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 244007936 bytes 2021-03-09T03:04:27.167 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 275085312 bytes 2021-03-09T03:04:27.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 381794304 bytes 2021-03-09T03:04:28.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 410845184 bytes 2021-03-09T03:04:28.412 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 439142400 bytes 2021-03-09T03:04:28.784 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 461634560 bytes 2021-03-09T03:04:29.283 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 556937216 bytes 2021-03-09T03:04:29.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 585742336 bytes 2021-03-09T03:04:30.027 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 586360832 bytes 2021-03-09T03:04:30.788 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 707050496 bytes 2021-03-09T03:04:31.185 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 707696640 bytes 2021-03-09T03:04:31.558 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 715365376 bytes 2021-03-09T03:04:31.961 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 747225088 bytes 2021-03-09T03:04:32.367 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 785803264 bytes 2021-03-09T03:04:32.730 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 806949888 bytes 2021-03-09T03:04:33.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 807549952 bytes 2021-03-09T03:04:33.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 868327424 bytes 2021-03-09T03:04:33.874 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 911202304 bytes 2021-03-09T03:04:34.188 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 918028288 bytes 2021-03-09T03:04:34.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 957822976 bytes 2021-03-09T03:04:34.961 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 980843520 bytes 2021-03-09T03:04:35.377 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 990607360 bytes 2021-03-09T03:04:35.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:04:35.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:04:35.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:04:35.607 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:04:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_24 2021-03-09T03:04:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_24 snapshot 128M 2021-03-09T03:04:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:35.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:35.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:35.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_24 2021-03-09T03:04:35.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:35.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:04:35.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:04:35.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:35.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_24 128M 2021-03-09T03:04:35.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:35.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:35.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:35.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:35.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_24 2021-03-09T03:04:35.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:35.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:04:35.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:04:35.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:04:35.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:35.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_24 2021-03-09T03:04:35.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:04:35.694 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_24 snapshot 2021-03-09T03:04:35.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:35.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:35.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_24 2021-03-09T03:04:35.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:04:35.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_24 snapshot 2021-03-09T03:04:36.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:04:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:04:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_24 2021-03-09T03:04:36.643 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_24 2021-03-09T03:04:36.644 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:04:36.645 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=10 2021-03-09T03:04:36.645 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:04:36.645 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_24 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:04:37.219 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 80663552 bytes 2021-03-09T03:04:37.589 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 108435456 bytes 2021-03-09T03:04:37.907 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 115135488 bytes 2021-03-09T03:04:38.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 199770112 bytes 2021-03-09T03:04:38.952 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 222510080 bytes 2021-03-09T03:04:39.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 275689472 bytes 2021-03-09T03:04:39.802 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 332229632 bytes 2021-03-09T03:04:40.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 439782400 bytes 2021-03-09T03:04:40.840 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 492637184 bytes 2021-03-09T03:04:41.182 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 498820096 bytes 2021-03-09T03:04:41.641 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 562841600 bytes 2021-03-09T03:04:42.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 604537856 bytes 2021-03-09T03:04:42.295 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T03:04:42.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T03:04:42.296 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T03:04:42.297 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T03:04:42.291+0000 7fed530dc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 29387) UID: 1000 2021-03-09T03:04:42.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 630579200 bytes 2021-03-09T03:04:42.697 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:04:42.849 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 662697984 bytes 2021-03-09T03:04:43.210 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 696646656 bytes 2021-03-09T03:04:43.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 731469824 bytes 2021-03-09T03:04:44.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 785326080 bytes 2021-03-09T03:04:44.565 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 859416576 bytes 2021-03-09T03:04:44.997 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 890642432 bytes 2021-03-09T03:04:45.719 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 937667584 bytes 2021-03-09T03:04:46.135 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 994727936 bytes 2021-03-09T03:04:46.591 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 1036743680 bytes 2021-03-09T03:04:46.653 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:04:46.653 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:04:46.654 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:04:46.654 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:04:46.654 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_25 2021-03-09T03:04:46.654 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_25 snapshot 128M 2021-03-09T03:04:46.655 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:46.655 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:46.655 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:46.655 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:46.655 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_25 2021-03-09T03:04:46.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:46.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:04:46.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:04:46.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_25 128M 2021-03-09T03:04:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:46.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:46.658 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_25 2021-03-09T03:04:46.658 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:46.658 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:04:46.658 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:04:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:04:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:04:46.659 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_25 2021-03-09T03:04:46.757 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:04:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_25 snapshot 2021-03-09T03:04:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_25 2021-03-09T03:04:46.758 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:04:46.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_25 snapshot 2021-03-09T03:04:47.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:04:47.647 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:04:47.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_25 2021-03-09T03:04:47.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:04:47.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:04:47.648 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_25 2021-03-09T03:04:47.649 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:04:47.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=14 2021-03-09T03:04:47.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:04:47.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_25 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:04:48.116 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 47346688 bytes 2021-03-09T03:04:48.519 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 82808832 bytes 2021-03-09T03:04:49.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 191013888 bytes 2021-03-09T03:04:49.404 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 201355264 bytes 2021-03-09T03:04:49.836 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 251680768 bytes 2021-03-09T03:04:50.172 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 273324032 bytes 2021-03-09T03:04:50.297 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T03:04:50.297 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:04:50.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:04:50.298 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T03:04:50.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T03:04:50.299 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T03:04:50.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T03:04:50.301 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T03:04:50.548 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 316627968 bytes 2021-03-09T03:04:51.115 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 365675520 bytes 2021-03-09T03:04:51.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 409063424 bytes 2021-03-09T03:04:51.911 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 448468992 bytes 2021-03-09T03:04:52.334 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 507829248 bytes 2021-03-09T03:04:52.755 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 555502592 bytes 2021-03-09T03:04:53.116 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 556195840 bytes 2021-03-09T03:04:53.502 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 575093760 bytes 2021-03-09T03:04:53.958 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 643705856 bytes 2021-03-09T03:04:54.409 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 711767040 bytes 2021-03-09T03:04:54.889 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 776432640 bytes 2021-03-09T03:04:55.338 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 834846720 bytes 2021-03-09T03:04:55.848 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 898331648 bytes 2021-03-09T03:04:56.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 961467392 bytes 2021-03-09T03:04:56.730 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 994906112 bytes 2021-03-09T03:04:57.259 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 1072795648 bytes 2021-03-09T03:04:57.639 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 1112719360 bytes 2021-03-09T03:04:58.102 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1160437760 bytes 2021-03-09T03:04:58.496 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1188508672 bytes 2021-03-09T03:04:58.951 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1243264000 bytes 2021-03-09T03:04:59.379 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1290665984 bytes 2021-03-09T03:04:59.679 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1302594560 bytes 2021-03-09T03:05:00.151 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1346443264 bytes 2021-03-09T03:05:00.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1384456192 bytes 2021-03-09T03:05:00.924 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1391583232 bytes 2021-03-09T03:05:01.365 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1449713664 bytes 2021-03-09T03:05:01.655 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:05:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:05:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:05:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:05:01.656 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_26 2021-03-09T03:05:01.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_26 snapshot 128M 2021-03-09T03:05:01.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:01.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:01.657 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:01.658 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:01.658 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_26 2021-03-09T03:05:01.658 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:01.658 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:01.659 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:05:01.659 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:01.659 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_26 128M 2021-03-09T03:05:01.659 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:01.659 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:01.660 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:01.660 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:01.660 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_26 2021-03-09T03:05:01.660 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:01.661 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:05:01.661 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:05:01.661 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:05:01.662 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:01.662 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_26 2021-03-09T03:05:01.742 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:05:01.742 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_26 snapshot 2021-03-09T03:05:01.742 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:01.743 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:01.743 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_26 2021-03-09T03:05:01.743 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:01.743 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_26 snapshot 2021-03-09T03:05:02.654 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:05:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:05:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_26 2021-03-09T03:05:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:02.689 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:02.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_26 2021-03-09T03:05:02.690 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:05:02.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=6 2021-03-09T03:05:02.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:05:02.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_26 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:05:03.190 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 43680768 bytes 2021-03-09T03:05:03.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 68338688 bytes 2021-03-09T03:05:04.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 129687552 bytes 2021-03-09T03:05:04.489 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 178668544 bytes 2021-03-09T03:05:04.835 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 202988544 bytes 2021-03-09T03:05:05.522 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 251709440 bytes 2021-03-09T03:05:05.912 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 302746624 bytes 2021-03-09T03:05:06.243 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 303365120 bytes 2021-03-09T03:05:06.702 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 348307456 bytes 2021-03-09T03:05:07.104 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 373868544 bytes 2021-03-09T03:05:07.595 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 404305920 bytes 2021-03-09T03:05:07.977 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 433315840 bytes 2021-03-09T03:05:08.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 478359552 bytes 2021-03-09T03:05:08.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:05:08.695 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:05:08.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:05:08.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:05:08.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_27 2021-03-09T03:05:08.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_27 snapshot 128M 2021-03-09T03:05:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:08.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_27 2021-03-09T03:05:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:05:08.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:08.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_27 128M 2021-03-09T03:05:08.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:08.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:08.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:08.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:08.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_27 2021-03-09T03:05:08.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:08.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:05:08.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:05:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:05:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_27 2021-03-09T03:05:08.791 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:05:08.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_27 snapshot 2021-03-09T03:05:08.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:08.792 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:08.793 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_27 2021-03-09T03:05:08.793 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:08.793 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_27 snapshot 2021-03-09T03:05:09.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:05:09.618 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:05:09.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_27 2021-03-09T03:05:09.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:09.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:09.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_27 2021-03-09T03:05:09.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:05:09.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=5 2021-03-09T03:05:09.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:05:09.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_27 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:05:10.215 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 43173888 bytes 2021-03-09T03:05:10.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 109211648 bytes 2021-03-09T03:05:11.075 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 129518592 bytes 2021-03-09T03:05:11.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 196298752 bytes 2021-03-09T03:05:11.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 205388800 bytes 2021-03-09T03:05:12.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 216632320 bytes 2021-03-09T03:05:12.972 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 240261120 bytes 2021-03-09T03:05:13.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 317858816 bytes 2021-03-09T03:05:13.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 325769216 bytes 2021-03-09T03:05:14.212 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 346715136 bytes 2021-03-09T03:05:14.626 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:05:14.627 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:05:14.627 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:05:14.627 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:05:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_28 2021-03-09T03:05:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_28 snapshot 128M 2021-03-09T03:05:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:14.628 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:14.629 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:14.629 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_28 2021-03-09T03:05:14.629 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:14.629 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:05:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_28 128M 2021-03-09T03:05:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:14.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:14.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:14.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:14.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_28 2021-03-09T03:05:14.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:05:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:05:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:05:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:14.632 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_28 2021-03-09T03:05:15.480 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:05:15.481 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_28 snapshot 2021-03-09T03:05:15.481 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:15.482 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:15.482 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_28 2021-03-09T03:05:15.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:15.483 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_28 snapshot 2021-03-09T03:05:16.066 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8925349442570376) > weight (0.25) 2021-03-09T03:05:16.067 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T03:05:16.067 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T03:05:16.068 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T03:05:16.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T03:05:16.063+0000 7f57c53d4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 29655) UID: 1000 2021-03-09T03:05:16.469 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:05:16.637 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:05:16.664 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:05:16.664 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_28 2021-03-09T03:05:16.664 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:16.665 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:16.665 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_28 2021-03-09T03:05:16.665 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:05:16.666 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=11 2021-03-09T03:05:16.667 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:05:16.667 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_28 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:05:17.135 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 4683776 bytes 2021-03-09T03:05:17.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 43719680 bytes 2021-03-09T03:05:17.848 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 65069056 bytes 2021-03-09T03:05:18.308 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 134845440 bytes 2021-03-09T03:05:18.716 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 176523264 bytes 2021-03-09T03:05:19.186 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 276165632 bytes 2021-03-09T03:05:19.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 313991168 bytes 2021-03-09T03:05:20.003 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 338098176 bytes 2021-03-09T03:05:20.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 344713216 bytes 2021-03-09T03:05:20.780 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 376306688 bytes 2021-03-09T03:05:21.136 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 399687680 bytes 2021-03-09T03:05:21.506 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 435258368 bytes 2021-03-09T03:05:21.895 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 471516160 bytes 2021-03-09T03:05:22.320 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 508059648 bytes 2021-03-09T03:05:22.837 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 585687040 bytes 2021-03-09T03:05:23.069 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T03:05:23.070 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:05:23.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:05:23.071 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T03:05:23.072 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T03:05:23.072 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T03:05:23.075 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T03:05:23.075 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T03:05:23.245 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 610316288 bytes 2021-03-09T03:05:23.809 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 710898688 bytes 2021-03-09T03:05:24.186 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 730427392 bytes 2021-03-09T03:05:24.924 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 814694400 bytes 2021-03-09T03:05:25.318 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 839001088 bytes 2021-03-09T03:05:25.823 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 914823168 bytes 2021-03-09T03:05:26.236 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 965173248 bytes 2021-03-09T03:05:26.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 1028820992 bytes 2021-03-09T03:05:27.239 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1068506112 bytes 2021-03-09T03:05:27.640 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1123436544 bytes 2021-03-09T03:05:27.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:05:27.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:05:27.681 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:05:27.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:05:27.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_29 2021-03-09T03:05:27.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_29 snapshot 128M 2021-03-09T03:05:27.682 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:27.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:27.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:27.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:27.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_29 2021-03-09T03:05:27.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:27.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:27.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:05:27.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:27.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_29 128M 2021-03-09T03:05:27.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:27.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:27.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:27.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:27.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_29 2021-03-09T03:05:27.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:27.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:05:27.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:05:27.686 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:05:27.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:27.687 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_29 2021-03-09T03:05:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:05:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_29 snapshot 2021-03-09T03:05:27.778 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:27.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:27.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_29 2021-03-09T03:05:27.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:27.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_29 snapshot 2021-03-09T03:05:28.011 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:05:28.043 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:05:28.044 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_29 2021-03-09T03:05:28.044 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:28.044 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:28.045 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_29 2021-03-09T03:05:28.045 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:05:28.046 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=9 2021-03-09T03:05:28.046 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:05:28.046 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_29 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:05:28.487 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 45543424 bytes 2021-03-09T03:05:28.900 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 70420480 bytes 2021-03-09T03:05:29.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 82138112 bytes 2021-03-09T03:05:29.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 143319040 bytes 2021-03-09T03:05:30.093 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 158273536 bytes 2021-03-09T03:05:30.425 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 161395712 bytes 2021-03-09T03:05:30.904 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 217295872 bytes 2021-03-09T03:05:31.271 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 255448064 bytes 2021-03-09T03:05:31.586 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 266968064 bytes 2021-03-09T03:05:32.128 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 332700672 bytes 2021-03-09T03:05:32.549 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 360724480 bytes 2021-03-09T03:05:32.910 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 384791552 bytes 2021-03-09T03:05:33.382 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 460627968 bytes 2021-03-09T03:05:33.878 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 503184384 bytes 2021-03-09T03:05:34.307 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 557025280 bytes 2021-03-09T03:05:34.788 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 611412992 bytes 2021-03-09T03:05:35.220 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 672078848 bytes 2021-03-09T03:05:35.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 690716672 bytes 2021-03-09T03:05:36.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 761167872 bytes 2021-03-09T03:05:36.820 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 815747072 bytes 2021-03-09T03:05:37.052 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:05:37.052 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:05:37.052 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:05:37.053 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:05:37.053 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_30 2021-03-09T03:05:37.053 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_30 snapshot 128M 2021-03-09T03:05:37.053 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:37.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:37.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:37.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:37.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_30 2021-03-09T03:05:37.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:37.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:37.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:05:37.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:37.055 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_30 128M 2021-03-09T03:05:37.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:37.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:37.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:37.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:37.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_30 2021-03-09T03:05:37.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:37.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:05:37.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:05:37.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:05:37.057 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:37.058 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_30 2021-03-09T03:05:37.147 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:05:37.147 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_30 snapshot 2021-03-09T03:05:37.148 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:37.148 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:37.148 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_30 2021-03-09T03:05:37.148 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:37.149 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_30 snapshot 2021-03-09T03:05:37.718 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:05:37.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:05:37.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_30 2021-03-09T03:05:37.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_30 2021-03-09T03:05:37.760 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:05:37.762 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=6 2021-03-09T03:05:37.762 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:05:37.762 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 6s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_30 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:05:38.242 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 47608832 bytes 2021-03-09T03:05:38.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 79137792 bytes 2021-03-09T03:05:39.114 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 121031680 bytes 2021-03-09T03:05:39.543 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 175647744 bytes 2021-03-09T03:05:39.893 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 176263168 bytes 2021-03-09T03:05:40.255 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 176864256 bytes 2021-03-09T03:05:40.749 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 229610496 bytes 2021-03-09T03:05:41.219 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 308034560 bytes 2021-03-09T03:05:41.592 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 338556928 bytes 2021-03-09T03:05:42.059 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 408628224 bytes 2021-03-09T03:05:42.517 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 458588160 bytes 2021-03-09T03:05:42.904 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 476692480 bytes 2021-03-09T03:05:43.304 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 505387008 bytes 2021-03-09T03:05:43.738 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 555160576 bytes 2021-03-09T03:05:43.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:05:43.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:05:43.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:05:43.767 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:05:43.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_31 2021-03-09T03:05:43.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_31 snapshot 128M 2021-03-09T03:05:43.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:43.768 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_31 2021-03-09T03:05:43.769 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:05:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_31 128M 2021-03-09T03:05:43.770 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:43.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:43.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:43.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:43.771 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_31 2021-03-09T03:05:43.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:43.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:05:43.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:05:43.772 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:05:43.773 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:43.773 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_31 2021-03-09T03:05:43.849 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:05:43.850 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_31 snapshot 2021-03-09T03:05:43.850 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:43.850 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:43.850 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_31 2021-03-09T03:05:43.851 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:43.851 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_31 snapshot 2021-03-09T03:05:44.576 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:05:44.608 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:05:44.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_31 2021-03-09T03:05:44.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:44.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:44.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_31 2021-03-09T03:05:44.610 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:05:44.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=5 2021-03-09T03:05:44.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:05:45.036 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_31 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:05:45.750 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 56093696 bytes 2021-03-09T03:05:46.138 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 78910464 bytes 2021-03-09T03:05:46.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 98604032 bytes 2021-03-09T03:05:46.930 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 150427648 bytes 2021-03-09T03:05:47.391 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 162203648 bytes 2021-03-09T03:05:47.974 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 215091200 bytes 2021-03-09T03:05:48.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 247072768 bytes 2021-03-09T03:05:48.840 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 291254272 bytes 2021-03-09T03:05:49.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 349728768 bytes 2021-03-09T03:05:49.748 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 384493568 bytes 2021-03-09T03:05:50.042 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:05:50.043 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:05:50.043 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:05:50.043 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:05:50.044 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_32 2021-03-09T03:05:50.044 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_32 snapshot 128M 2021-03-09T03:05:50.044 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:50.044 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:50.044 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_32 2021-03-09T03:05:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:50.045 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:05:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_32 128M 2021-03-09T03:05:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:50.046 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_32 2021-03-09T03:05:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:50.047 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:05:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:05:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:05:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:05:50.048 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_32 2021-03-09T03:05:50.135 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:05:50.135 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_32 snapshot 2021-03-09T03:05:50.135 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:50.135 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:50.136 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_32 2021-03-09T03:05:50.136 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:05:50.136 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_32 snapshot 2021-03-09T03:05:50.623 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:05:50.668 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:05:50.669 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_32 2021-03-09T03:05:50.669 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:05:50.669 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:05:50.669 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_32 2021-03-09T03:05:50.669 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:05:50.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=9 2021-03-09T03:05:50.784 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:05:51.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_32 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:05:51.988 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 111431680 bytes 2021-03-09T03:05:52.375 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 154397696 bytes 2021-03-09T03:05:52.773 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 172110848 bytes 2021-03-09T03:05:53.192 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 211305472 bytes 2021-03-09T03:05:53.661 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 249428992 bytes 2021-03-09T03:05:54.094 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 267815936 bytes 2021-03-09T03:05:54.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 302437376 bytes 2021-03-09T03:05:54.995 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 389743616 bytes 2021-03-09T03:05:55.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 439618560 bytes 2021-03-09T03:05:55.891 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 452364288 bytes 2021-03-09T03:05:56.345 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 493790208 bytes 2021-03-09T03:05:56.798 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 525541376 bytes 2021-03-09T03:05:57.239 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 565678080 bytes 2021-03-09T03:05:57.629 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 590842880 bytes 2021-03-09T03:05:57.990 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 612384768 bytes 2021-03-09T03:05:58.414 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 648332288 bytes 2021-03-09T03:05:58.750 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 668791808 bytes 2021-03-09T03:05:59.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 704318464 bytes 2021-03-09T03:05:59.866 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 828368896 bytes 2021-03-09T03:06:00.304 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 869193728 bytes 2021-03-09T03:06:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:06:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:06:00.353 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:06:00.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:06:00.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_33 2021-03-09T03:06:00.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_33 snapshot 128M 2021-03-09T03:06:00.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:00.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:00.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:00.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:00.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_33 2021-03-09T03:06:00.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:00.355 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:00.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:06:00.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:00.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_33 128M 2021-03-09T03:06:00.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_33 2021-03-09T03:06:00.357 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:06:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:06:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:06:00.358 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:00.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_33 2021-03-09T03:06:00.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:06:00.484 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_33 snapshot 2021-03-09T03:06:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_33 2021-03-09T03:06:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:00.485 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_33 snapshot 2021-03-09T03:06:01.700 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:06:01.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:06:01.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_33 2021-03-09T03:06:01.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:01.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:01.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_33 2021-03-09T03:06:01.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:06:01.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=10 2021-03-09T03:06:01.736 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:06:02.375 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 10s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_33 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:06:02.896 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 26673152 bytes 2021-03-09T03:06:03.402 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 109030400 bytes 2021-03-09T03:06:03.728 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 109650944 bytes 2021-03-09T03:06:04.170 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 177872896 bytes 2021-03-09T03:06:04.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 230730752 bytes 2021-03-09T03:06:05.022 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 263226368 bytes 2021-03-09T03:06:05.444 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 296018944 bytes 2021-03-09T03:06:05.957 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 342984704 bytes 2021-03-09T03:06:06.413 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 412657664 bytes 2021-03-09T03:06:06.843 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 454517760 bytes 2021-03-09T03:06:07.170 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 498823168 bytes 2021-03-09T03:06:07.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 534041600 bytes 2021-03-09T03:06:07.914 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 571700224 bytes 2021-03-09T03:06:08.394 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 651891712 bytes 2021-03-09T03:06:08.827 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 716338176 bytes 2021-03-09T03:06:09.269 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 804490240 bytes 2021-03-09T03:06:09.683 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 852640768 bytes 2021-03-09T03:06:10.130 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 919304192 bytes 2021-03-09T03:06:10.486 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 942482432 bytes 2021-03-09T03:06:10.840 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 972535808 bytes 2021-03-09T03:06:11.254 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 1011628032 bytes 2021-03-09T03:06:11.601 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 1043978240 bytes 2021-03-09T03:06:11.994 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 1059425280 bytes 2021-03-09T03:06:12.297 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1074225152 bytes 2021-03-09T03:06:12.382 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:06:12.383 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:06:12.383 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:06:12.383 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:06:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_34 2021-03-09T03:06:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_34 snapshot 128M 2021-03-09T03:06:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:12.384 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:12.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:12.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_34 2021-03-09T03:06:12.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:12.385 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:12.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:06:12.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:12.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_34 128M 2021-03-09T03:06:12.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:12.386 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:12.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:12.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:12.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_34 2021-03-09T03:06:12.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:12.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:06:12.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:06:12.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:06:12.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:12.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_34 2021-03-09T03:06:12.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:06:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_34 snapshot 2021-03-09T03:06:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:12.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_34 2021-03-09T03:06:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:12.459 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_34 snapshot 2021-03-09T03:06:12.675 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:06:12.705 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:06:12.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_34 2021-03-09T03:06:12.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:12.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:12.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_34 2021-03-09T03:06:12.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:06:12.708 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=9 2021-03-09T03:06:12.708 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:06:12.708 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 9s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_34 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:06:13.078 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 2089984 bytes 2021-03-09T03:06:13.431 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 49214464 bytes 2021-03-09T03:06:13.807 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 107997184 bytes 2021-03-09T03:06:14.319 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 222632960 bytes 2021-03-09T03:06:14.728 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 261576704 bytes 2021-03-09T03:06:15.197 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 342517760 bytes 2021-03-09T03:06:15.796 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 400496640 bytes 2021-03-09T03:06:16.097 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 416869376 bytes 2021-03-09T03:06:16.500 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 480590848 bytes 2021-03-09T03:06:16.934 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 529467392 bytes 2021-03-09T03:06:17.327 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 581607424 bytes 2021-03-09T03:06:17.750 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 642656256 bytes 2021-03-09T03:06:18.175 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 703002624 bytes 2021-03-09T03:06:18.570 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 756131840 bytes 2021-03-09T03:06:18.848 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 757880832 bytes 2021-03-09T03:06:19.182 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 795621376 bytes 2021-03-09T03:06:19.559 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 813796352 bytes 2021-03-09T03:06:19.970 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 872893440 bytes 2021-03-09T03:06:20.495 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 963436544 bytes 2021-03-09T03:06:20.962 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 1037720576 bytes 2021-03-09T03:06:21.311 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 1056120832 bytes 2021-03-09T03:06:21.630 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 1072386048 bytes 2021-03-09T03:06:21.712 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:06:21.712 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:06:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:06:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:06:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_35 2021-03-09T03:06:21.713 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_35 snapshot 128M 2021-03-09T03:06:21.714 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:21.714 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:21.714 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:21.714 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:21.715 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_35 2021-03-09T03:06:21.715 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:21.715 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:21.715 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:06:21.715 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:21.716 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_35 128M 2021-03-09T03:06:21.716 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:21.716 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:21.716 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:21.716 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:21.717 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_35 2021-03-09T03:06:21.717 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:21.717 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:06:21.717 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:06:21.718 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:06:21.718 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:21.718 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_35 2021-03-09T03:06:21.805 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:06:21.805 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_35 snapshot 2021-03-09T03:06:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:21.806 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_35 2021-03-09T03:06:21.807 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:21.807 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_35 snapshot 2021-03-09T03:06:22.845 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:06:22.877 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:06:22.878 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_35 2021-03-09T03:06:22.878 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:22.878 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:22.878 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_35 2021-03-09T03:06:22.879 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:06:22.879 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=11 2021-03-09T03:06:22.880 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:06:22.880 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 11s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_35 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:06:23.304 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 31973376 bytes 2021-03-09T03:06:23.727 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 52532224 bytes 2021-03-09T03:06:24.094 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 71231488 bytes 2021-03-09T03:06:24.477 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 106101760 bytes 2021-03-09T03:06:24.909 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 151387136 bytes 2021-03-09T03:06:25.241 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 178294784 bytes 2021-03-09T03:06:25.629 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 197916672 bytes 2021-03-09T03:06:26.169 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 232906752 bytes 2021-03-09T03:06:26.634 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 276638720 bytes 2021-03-09T03:06:26.980 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 286098432 bytes 2021-03-09T03:06:27.289 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 301145088 bytes 2021-03-09T03:06:27.615 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 311831552 bytes 2021-03-09T03:06:28.105 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 381284352 bytes 2021-03-09T03:06:28.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 439923712 bytes 2021-03-09T03:06:28.947 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 490646528 bytes 2021-03-09T03:06:29.246 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 491499520 bytes 2021-03-09T03:06:29.667 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 543777792 bytes 2021-03-09T03:06:30.113 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 603482112 bytes 2021-03-09T03:06:30.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5124060529410602) > weight (0.25) 2021-03-09T03:06:30.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7163075175039808) > weight (0.25) 2021-03-09T03:06:30.303 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4912738209574208) > weight (0.25) 2021-03-09T03:06:30.303 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4269014566461725) > weight (0.25) 2021-03-09T03:06:30.303 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T03:06:30.536 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 651260928 bytes 2021-03-09T03:06:31.063 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 673135616 bytes 2021-03-09T03:06:31.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 673764352 bytes 2021-03-09T03:06:31.783 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 732091392 bytes 2021-03-09T03:06:32.173 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 761177088 bytes 2021-03-09T03:06:32.525 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 785840128 bytes 2021-03-09T03:06:32.973 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 847409152 bytes 2021-03-09T03:06:33.280 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 853506048 bytes 2021-03-09T03:06:33.753 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 915965952 bytes 2021-03-09T03:06:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:06:33.884 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:06:33.885 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:06:33.885 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:06:33.885 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_36 2021-03-09T03:06:33.885 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_36 snapshot 128M 2021-03-09T03:06:33.886 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:33.886 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:33.886 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:33.886 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:33.887 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_36 2021-03-09T03:06:33.887 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:33.887 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:33.887 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:06:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_36 128M 2021-03-09T03:06:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:33.888 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_36 2021-03-09T03:06:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:06:33.889 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:06:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:06:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:33.890 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_36 2021-03-09T03:06:33.966 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:06:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_36 snapshot 2021-03-09T03:06:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:33.967 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_36 2021-03-09T03:06:33.968 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:33.968 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_36 snapshot 2021-03-09T03:06:34.581 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:06:34.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:06:34.611 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_36 2021-03-09T03:06:34.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:34.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:34.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_36 2021-03-09T03:06:34.612 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:06:34.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=8 2021-03-09T03:06:34.613 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:06:34.614 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_36 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:06:35.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 18673664 bytes 2021-03-09T03:06:35.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 62225408 bytes 2021-03-09T03:06:35.918 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 111825920 bytes 2021-03-09T03:06:36.268 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 112441344 bytes 2021-03-09T03:06:36.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 149663744 bytes 2021-03-09T03:06:37.009 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 159493120 bytes 2021-03-09T03:06:37.443 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 208718848 bytes 2021-03-09T03:06:37.787 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 211960832 bytes 2021-03-09T03:06:38.117 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 219778048 bytes 2021-03-09T03:06:38.660 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 324883456 bytes 2021-03-09T03:06:39.124 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 404237312 bytes 2021-03-09T03:06:39.640 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 459518976 bytes 2021-03-09T03:06:40.087 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 514339840 bytes 2021-03-09T03:06:40.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 561162240 bytes 2021-03-09T03:06:40.874 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 584802304 bytes 2021-03-09T03:06:41.248 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 617965568 bytes 2021-03-09T03:06:41.679 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 658980864 bytes 2021-03-09T03:06:42.148 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 744398848 bytes 2021-03-09T03:06:42.545 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 784457728 bytes 2021-03-09T03:06:42.618 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:06:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:06:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:06:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:06:42.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_37 2021-03-09T03:06:42.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_37 snapshot 128M 2021-03-09T03:06:42.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:42.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:42.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:42.620 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:42.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_37 2021-03-09T03:06:42.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:42.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:42.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:06:42.621 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:42.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_37 128M 2021-03-09T03:06:42.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:42.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:42.622 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_37 2021-03-09T03:06:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:06:42.623 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:06:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:06:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:42.624 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_37 2021-03-09T03:06:42.862 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:06:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_37 snapshot 2021-03-09T03:06:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:42.863 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_37 2021-03-09T03:06:42.864 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:42.864 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_37 snapshot 2021-03-09T03:06:43.461 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:06:43.503 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:06:43.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_37 2021-03-09T03:06:43.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:43.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:43.504 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_37 2021-03-09T03:06:43.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:06:43.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=13 2021-03-09T03:06:43.506 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:06:43.506 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 13s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_37 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:06:43.989 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 59444224 bytes 2021-03-09T03:06:44.437 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 119475200 bytes 2021-03-09T03:06:44.782 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 136238080 bytes 2021-03-09T03:06:45.210 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 181259264 bytes 2021-03-09T03:06:45.606 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 211629056 bytes 2021-03-09T03:06:46.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 263677952 bytes 2021-03-09T03:06:46.373 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 276167680 bytes 2021-03-09T03:06:46.815 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 325809152 bytes 2021-03-09T03:06:47.475 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 401856512 bytes 2021-03-09T03:06:47.799 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 413858816 bytes 2021-03-09T03:06:48.250 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 489166848 bytes 2021-03-09T03:06:48.673 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 515372032 bytes 2021-03-09T03:06:49.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 523051008 bytes 2021-03-09T03:06:49.349 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 537712640 bytes 2021-03-09T03:06:49.693 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 560652288 bytes 2021-03-09T03:06:50.127 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 621690880 bytes 2021-03-09T03:06:50.540 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 700796928 bytes 2021-03-09T03:06:51.176 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 760633344 bytes 2021-03-09T03:06:51.691 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 777861120 bytes 2021-03-09T03:06:52.084 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 799215616 bytes 2021-03-09T03:06:52.469 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 821358592 bytes 2021-03-09T03:06:52.932 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 897458176 bytes 2021-03-09T03:06:53.274 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 909789184 bytes 2021-03-09T03:06:53.627 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 946506752 bytes 2021-03-09T03:06:53.980 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1003675648 bytes 2021-03-09T03:06:54.316 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1064773632 bytes 2021-03-09T03:06:54.603 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1102891008 bytes 2021-03-09T03:06:54.946 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1158775808 bytes 2021-03-09T03:06:55.295 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1224248320 bytes 2021-03-09T03:06:55.717 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1315218432 bytes 2021-03-09T03:06:56.131 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1334037504 bytes 2021-03-09T03:06:56.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1384396800 bytes 2021-03-09T03:06:56.510 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:06:56.510 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:06:56.511 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:06:56.511 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:06:56.511 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_38 2021-03-09T03:06:56.511 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_38 snapshot 128M 2021-03-09T03:06:56.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:56.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:56.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:56.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:56.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_38 2021-03-09T03:06:56.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:56.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:56.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:06:56.513 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:56.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_38 128M 2021-03-09T03:06:56.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:56.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:56.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:56.514 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:56.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_38 2021-03-09T03:06:56.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:56.515 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:06:56.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:06:56.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:06:56.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:06:56.516 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_38 2021-03-09T03:06:56.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:06:56.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_38 snapshot 2021-03-09T03:06:56.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:56.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:56.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_38 2021-03-09T03:06:56.583 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:06:56.584 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_38 snapshot 2021-03-09T03:06:57.688 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:06:57.723 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:06:57.723 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_38 2021-03-09T03:06:57.723 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:06:57.724 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:06:57.724 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_38 2021-03-09T03:06:57.724 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:06:57.725 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=8 2021-03-09T03:06:57.725 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:06:57.725 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 8s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_38 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:06:58.099 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 40268800 bytes 2021-03-09T03:06:58.438 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 92410880 bytes 2021-03-09T03:06:58.835 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 161123328 bytes 2021-03-09T03:06:59.125 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 190286848 bytes 2021-03-09T03:06:59.512 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 217198592 bytes 2021-03-09T03:06:59.853 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 249632768 bytes 2021-03-09T03:07:00.208 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 312161280 bytes 2021-03-09T03:07:00.588 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 378113024 bytes 2021-03-09T03:07:00.965 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 419817472 bytes 2021-03-09T03:07:01.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 424194048 bytes 2021-03-09T03:07:01.791 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 455673856 bytes 2021-03-09T03:07:02.267 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 567931904 bytes 2021-03-09T03:07:02.638 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 624139264 bytes 2021-03-09T03:07:02.906 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 632922112 bytes 2021-03-09T03:07:03.331 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 687095808 bytes 2021-03-09T03:07:03.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 769705984 bytes 2021-03-09T03:07:03.991 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 786364416 bytes 2021-03-09T03:07:04.442 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 876085248 bytes 2021-03-09T03:07:04.745 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 917925888 bytes 2021-03-09T03:07:05.073 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 956016640 bytes 2021-03-09T03:07:05.389 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 996066304 bytes 2021-03-09T03:07:05.633 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 1005237248 bytes 2021-03-09T03:07:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:07:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:07:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:07:05.731 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:07:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_39 2021-03-09T03:07:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_39 snapshot 128M 2021-03-09T03:07:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:05.732 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_39 2021-03-09T03:07:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:05.733 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:07:05.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:07:05.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:05.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_39 128M 2021-03-09T03:07:05.734 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_39 2021-03-09T03:07:05.735 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:05.736 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:07:05.736 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:07:05.736 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:07:05.737 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:05.737 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_39 2021-03-09T03:07:05.814 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:07:05.815 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_39 snapshot 2021-03-09T03:07:05.815 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:05.815 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:05.816 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_39 2021-03-09T03:07:05.816 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:07:05.816 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_39 snapshot 2021-03-09T03:07:06.041 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:07:06.076 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:07:06.076 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_39 2021-03-09T03:07:06.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:06.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:06.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_39 2021-03-09T03:07:06.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:07:06.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=7 2021-03-09T03:07:06.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:07:06.079 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 7s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_39 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:07:06.747 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 69155840 bytes 2021-03-09T03:07:07.050 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 105461760 bytes 2021-03-09T03:07:07.332 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 128708608 bytes 2021-03-09T03:07:07.668 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 195792896 bytes 2021-03-09T03:07:07.896 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 218795008 bytes 2021-03-09T03:07:08.239 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 285095936 bytes 2021-03-09T03:07:08.701 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 384190464 bytes 2021-03-09T03:07:09.035 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 439632896 bytes 2021-03-09T03:07:09.315 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 470699008 bytes 2021-03-09T03:07:09.535 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 483737600 bytes 2021-03-09T03:07:09.939 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 579171328 bytes 2021-03-09T03:07:10.175 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 611886080 bytes 2021-03-09T03:07:10.580 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 675060736 bytes 2021-03-09T03:07:10.873 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 721142784 bytes 2021-03-09T03:07:11.258 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 787179520 bytes 2021-03-09T03:07:11.665 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 838720512 bytes 2021-03-09T03:07:12.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8503513657705698) > weight (0.25) 2021-03-09T03:07:12.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.811292803392224) > weight (0.25) 2021-03-09T03:07:12.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.39177623001273554) > weight (0.25) 2021-03-09T03:07:12.078 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5010325187527952) > weight (0.25) 2021-03-09T03:07:12.078 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T03:07:12.093 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 949166080 bytes 2021-03-09T03:07:12.582 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 1013304320 bytes 2021-03-09T03:07:12.939 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 1059737600 bytes 2021-03-09T03:07:13.085 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:07:13.085 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:07:13.085 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:07:13.086 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:07:13.086 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_40 2021-03-09T03:07:13.086 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_40 snapshot 128M 2021-03-09T03:07:13.086 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:13.086 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_40 2021-03-09T03:07:13.087 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:07:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:07:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_40 128M 2021-03-09T03:07:13.088 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:13.089 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:13.089 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:13.089 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:13.089 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_40 2021-03-09T03:07:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:07:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:07:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:07:13.090 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:13.091 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_40 2021-03-09T03:07:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:07:13.156 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_40 snapshot 2021-03-09T03:07:13.157 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:13.157 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:13.157 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_40 2021-03-09T03:07:13.158 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:07:13.158 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_40 snapshot 2021-03-09T03:07:13.684 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:07:13.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:07:13.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_40 2021-03-09T03:07:13.696 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:13.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:13.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_40 2021-03-09T03:07:13.697 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:07:13.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=14 2021-03-09T03:07:13.698 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:07:13.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_40 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:07:14.036 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 27539456 bytes 2021-03-09T03:07:14.323 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 56680448 bytes 2021-03-09T03:07:14.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 65944576 bytes 2021-03-09T03:07:14.973 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 135236608 bytes 2021-03-09T03:07:15.298 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 168409088 bytes 2021-03-09T03:07:15.604 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 189476864 bytes 2021-03-09T03:07:15.971 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 241799168 bytes 2021-03-09T03:07:16.247 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 271590400 bytes 2021-03-09T03:07:16.720 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 272216064 bytes 2021-03-09T03:07:17.011 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 307803136 bytes 2021-03-09T03:07:17.270 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 313078784 bytes 2021-03-09T03:07:17.726 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 370334720 bytes 2021-03-09T03:07:18.024 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 382619648 bytes 2021-03-09T03:07:18.369 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 423957504 bytes 2021-03-09T03:07:18.705 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 465510400 bytes 2021-03-09T03:07:18.983 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 489322496 bytes 2021-03-09T03:07:19.309 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 510125056 bytes 2021-03-09T03:07:19.618 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 535377920 bytes 2021-03-09T03:07:19.921 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 548444160 bytes 2021-03-09T03:07:20.245 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 617695232 bytes 2021-03-09T03:07:20.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 653487104 bytes 2021-03-09T03:07:20.894 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 686793728 bytes 2021-03-09T03:07:21.249 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 730847232 bytes 2021-03-09T03:07:21.676 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 776807424 bytes 2021-03-09T03:07:22.043 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 817820672 bytes 2021-03-09T03:07:22.401 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 860710912 bytes 2021-03-09T03:07:22.745 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 898353152 bytes 2021-03-09T03:07:23.132 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 954529792 bytes 2021-03-09T03:07:23.509 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1015794688 bytes 2021-03-09T03:07:23.899 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1099569152 bytes 2021-03-09T03:07:24.247 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1123116032 bytes 2021-03-09T03:07:24.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1195454464 bytes 2021-03-09T03:07:25.003 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1256187904 bytes 2021-03-09T03:07:25.283 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1276496896 bytes 2021-03-09T03:07:25.659 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1328487424 bytes 2021-03-09T03:07:25.955 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1362360320 bytes 2021-03-09T03:07:26.471 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1461633024 bytes 2021-03-09T03:07:26.949 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1503441920 bytes 2021-03-09T03:07:27.216 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1518072832 bytes 2021-03-09T03:07:27.569 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4000 IOs, 1552614400 bytes 2021-03-09T03:07:27.704 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:07:27.704 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:07:27.704 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:07:27.705 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:07:27.705 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_41 2021-03-09T03:07:27.705 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_41 snapshot 128M 2021-03-09T03:07:27.705 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:27.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:27.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:27.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:27.706 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_41 2021-03-09T03:07:27.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:27.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:07:27.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:07:27.707 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:27.708 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_41 128M 2021-03-09T03:07:27.708 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:27.708 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:27.708 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:27.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:27.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_41 2021-03-09T03:07:27.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:27.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:07:27.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:07:27.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:07:27.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:27.710 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_41 2021-03-09T03:07:27.775 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:07:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_41 snapshot 2021-03-09T03:07:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_41 2021-03-09T03:07:27.776 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:07:27.777 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_41 snapshot 2021-03-09T03:07:28.017 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:07:28.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:07:28.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_41 2021-03-09T03:07:28.060 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:28.061 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:28.061 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_41 2021-03-09T03:07:28.061 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:07:28.063 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=14 2021-03-09T03:07:28.063 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:07:28.063 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_41 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:07:28.458 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 39422976 bytes 2021-03-09T03:07:28.761 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 72334336 bytes 2021-03-09T03:07:29.134 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 143399936 bytes 2021-03-09T03:07:29.440 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 188564480 bytes 2021-03-09T03:07:29.709 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 196466688 bytes 2021-03-09T03:07:30.056 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 240262144 bytes 2021-03-09T03:07:30.359 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 274242560 bytes 2021-03-09T03:07:30.857 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 396725248 bytes 2021-03-09T03:07:31.190 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 446310400 bytes 2021-03-09T03:07:31.560 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 508244992 bytes 2021-03-09T03:07:32.200 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 572782592 bytes 2021-03-09T03:07:32.505 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 626105344 bytes 2021-03-09T03:07:32.778 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 646465536 bytes 2021-03-09T03:07:33.110 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 678309888 bytes 2021-03-09T03:07:33.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 714292224 bytes 2021-03-09T03:07:33.728 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 728059904 bytes 2021-03-09T03:07:34.038 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 777398272 bytes 2021-03-09T03:07:34.344 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 786747392 bytes 2021-03-09T03:07:34.741 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 857356288 bytes 2021-03-09T03:07:35.077 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 874527744 bytes 2021-03-09T03:07:35.307 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 900588544 bytes 2021-03-09T03:07:35.718 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 971298816 bytes 2021-03-09T03:07:35.967 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 971910144 bytes 2021-03-09T03:07:36.312 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1024307200 bytes 2021-03-09T03:07:36.663 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1064023040 bytes 2021-03-09T03:07:37.062 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1116455936 bytes 2021-03-09T03:07:37.409 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1159380992 bytes 2021-03-09T03:07:37.672 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1164013568 bytes 2021-03-09T03:07:38.001 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1197558784 bytes 2021-03-09T03:07:38.265 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1198213120 bytes 2021-03-09T03:07:38.596 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1239374848 bytes 2021-03-09T03:07:39.004 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1300901888 bytes 2021-03-09T03:07:39.302 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1313617920 bytes 2021-03-09T03:07:39.639 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1352442880 bytes 2021-03-09T03:07:39.977 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1396605952 bytes 2021-03-09T03:07:40.296 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1417607168 bytes 2021-03-09T03:07:40.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1440971776 bytes 2021-03-09T03:07:41.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1513047040 bytes 2021-03-09T03:07:41.447 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1591447552 bytes 2021-03-09T03:07:41.765 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4000 IOs, 1654307840 bytes 2021-03-09T03:07:42.069 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:07:42.069 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:07:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:07:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:07:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_42 2021-03-09T03:07:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_42 snapshot 128M 2021-03-09T03:07:42.070 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:42.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:42.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:42.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:42.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_42 2021-03-09T03:07:42.071 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:07:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:07:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:42.072 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_42 128M 2021-03-09T03:07:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_42 2021-03-09T03:07:42.073 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:07:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:07:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:07:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:42.074 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_42 2021-03-09T03:07:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:07:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_42 snapshot 2021-03-09T03:07:42.137 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_42 2021-03-09T03:07:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:07:42.138 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_42 snapshot 2021-03-09T03:07:42.356 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:07:42.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:07:42.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_42 2021-03-09T03:07:42.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:42.387 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_42 2021-03-09T03:07:42.388 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:07:42.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=14 2021-03-09T03:07:42.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:07:42.390 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 14s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_42 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:07:42.685 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 15415296 bytes 2021-03-09T03:07:43.025 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 64705536 bytes 2021-03-09T03:07:43.279 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 82854912 bytes 2021-03-09T03:07:43.663 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 173492224 bytes 2021-03-09T03:07:43.994 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 236693504 bytes 2021-03-09T03:07:44.335 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 288832512 bytes 2021-03-09T03:07:44.579 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 289437696 bytes 2021-03-09T03:07:44.913 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 339065856 bytes 2021-03-09T03:07:45.189 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 385719296 bytes 2021-03-09T03:07:45.598 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 451441664 bytes 2021-03-09T03:07:46.016 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 514074624 bytes 2021-03-09T03:07:46.405 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 595469312 bytes 2021-03-09T03:07:46.705 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 637614080 bytes 2021-03-09T03:07:47.136 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 670523392 bytes 2021-03-09T03:07:47.377 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 678198272 bytes 2021-03-09T03:07:47.653 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 699959296 bytes 2021-03-09T03:07:48.076 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 790197248 bytes 2021-03-09T03:07:48.460 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 864376832 bytes 2021-03-09T03:07:48.748 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1900 IOs, 904505344 bytes 2021-03-09T03:07:49.003 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2000 IOs, 930546688 bytes 2021-03-09T03:07:49.156 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2100 IOs, 934789120 bytes 2021-03-09T03:07:49.354 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2200 IOs, 964355072 bytes 2021-03-09T03:07:49.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2300 IOs, 1080359936 bytes 2021-03-09T03:07:50.004 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2400 IOs, 1124289536 bytes 2021-03-09T03:07:50.288 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2500 IOs, 1188581376 bytes 2021-03-09T03:07:50.518 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2600 IOs, 1220842496 bytes 2021-03-09T03:07:50.716 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2700 IOs, 1221480448 bytes 2021-03-09T03:07:51.054 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2800 IOs, 1299382272 bytes 2021-03-09T03:07:51.379 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:2900 IOs, 1379391488 bytes 2021-03-09T03:07:51.699 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3000 IOs, 1453868032 bytes 2021-03-09T03:07:51.843 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3100 IOs, 1455406080 bytes 2021-03-09T03:07:52.292 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3200 IOs, 1539355648 bytes 2021-03-09T03:07:52.619 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3300 IOs, 1613172736 bytes 2021-03-09T03:07:52.935 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3400 IOs, 1659932672 bytes 2021-03-09T03:07:53.154 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3500 IOs, 1689041920 bytes 2021-03-09T03:07:53.530 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3600 IOs, 1786663936 bytes 2021-03-09T03:07:53.861 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3700 IOs, 1847397376 bytes 2021-03-09T03:07:54.211 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3800 IOs, 1938725888 bytes 2021-03-09T03:07:54.449 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:3900 IOs, 1975547904 bytes 2021-03-09T03:07:54.759 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4000 IOs, 2053884928 bytes 2021-03-09T03:07:54.944 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4100 IOs, 2075598848 bytes 2021-03-09T03:07:55.213 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4200 IOs, 2125286400 bytes 2021-03-09T03:07:55.417 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4300 IOs, 2150252544 bytes 2021-03-09T03:07:55.636 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4400 IOs, 2185639936 bytes 2021-03-09T03:07:55.831 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4500 IOs, 2203587584 bytes 2021-03-09T03:07:56.093 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:4600 IOs, 2251117568 bytes 2021-03-09T03:07:56.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ error_code=124 2021-03-09T03:07:56.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set -e 2021-03-09T03:07:56.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ 124 -eq 124 ] 2021-03-09T03:07:56.395 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ return 0 2021-03-09T03:07:56.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ image=image_43 2021-03-09T03:07:56.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image_and_enable_mirror cluster2 mirror image_43 snapshot 128M 2021-03-09T03:07:56.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:56.396 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_43 2021-03-09T03:07:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:56.397 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:07:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n snapshot ] 2021-03-09T03:07:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ create_image cluster2 mirror image_43 128M 2021-03-09T03:07:56.398 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:56.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:56.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:56.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:56.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_43 2021-03-09T03:07:56.399 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:56.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local size=128 2021-03-09T03:07:56.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n 128M ] 2021-03-09T03:07:56.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ size=128M 2021-03-09T03:07:56.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ shift 2021-03-09T03:07:56.400 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster cluster2 create --size 128M --image-feature layering,exclusive-lock mirror/image_43 2021-03-09T03:07:56.455 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ image = image ] 2021-03-09T03:07:56.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ enable_mirror cluster2 mirror image_43 snapshot 2021-03-09T03:07:56.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:56.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:56.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_43 2021-03-09T03:07:56.456 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local mode=snapshot 2021-03-09T03:07:56.457 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ rbd --cluster=cluster2 mirror image enable mirror/image_43 snapshot 2021-03-09T03:07:56.631 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:Mirroring enabled 2021-03-09T03:07:56.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ [ -n ] 2021-03-09T03:07:56.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ stress_write_image cluster2 mirror image_43 2021-03-09T03:07:56.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local cluster=cluster2 2021-03-09T03:07:56.650 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local pool=mirror 2021-03-09T03:07:56.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local image=image_43 2021-03-09T03:07:56.651 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ awk BEGIN {srand(); print int(10 * rand()) + 5} 2021-03-09T03:07:56.652 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ local duration=5 2021-03-09T03:07:56.652 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ set +e 2021-03-09T03:07:56.652 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stderr:+ timeout 5s ceph_test_rbd_mirror_random_write --cluster cluster2 mirror image_43 --debug-rbd=20 --debug-journaler=20 2021-03-09T03:07:57.037 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:100 IOs, 68918272 bytes 2021-03-09T03:07:57.339 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:200 IOs, 129313792 bytes 2021-03-09T03:07:57.676 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:300 IOs, 193239040 bytes 2021-03-09T03:07:57.873 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:400 IOs, 215786496 bytes 2021-03-09T03:07:58.083 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:500 IOs, 243758080 bytes 2021-03-09T03:07:58.288 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:600 IOs, 268707840 bytes 2021-03-09T03:07:58.470 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:700 IOs, 289906688 bytes 2021-03-09T03:07:58.779 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:800 IOs, 362770432 bytes 2021-03-09T03:07:58.969 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:900 IOs, 363403264 bytes 2021-03-09T03:07:59.233 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1000 IOs, 413826048 bytes 2021-03-09T03:07:59.609 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1100 IOs, 506753024 bytes 2021-03-09T03:07:59.829 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1200 IOs, 530412544 bytes 2021-03-09T03:08:00.118 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1300 IOs, 594067456 bytes 2021-03-09T03:08:00.454 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1400 IOs, 655772672 bytes 2021-03-09T03:08:00.766 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1500 IOs, 719090688 bytes 2021-03-09T03:08:00.941 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1600 IOs, 730032128 bytes 2021-03-09T03:08:01.174 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1700 IOs, 762372096 bytes 2021-03-09T03:08:01.508 INFO:tasks.workunit.cluster1.client.mirror.smithi144.stdout:1800 IOs, 837998592 bytes 2021-03-09T03:08:27.304 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.639033872025186) > weight (0.25) 2021-03-09T03:08:27.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6583706579939838) > weight (0.25) 2021-03-09T03:08:27.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8448607487339737) > weight (0.25) 2021-03-09T03:08:27.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7858789411247679) > weight (0.25) 2021-03-09T03:08:27.305 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T03:09:11.079 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T03:09:11.080 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T03:09:11.081 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T03:09:11.081 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T03:09:11.075+0000 7f5e86d66700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 24345) UID: 1000 2021-03-09T03:09:11.282 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:09:14.081 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T03:09:14.082 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:09:14.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:09:14.082 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T03:09:14.083 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T03:09:14.083 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T03:09:14.086 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T03:09:14.086 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T03:09:21.094 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:09:21.086+0000 7f901a51c700 -1 reset not still connected to 0x5588e7f44700 2021-03-09T03:09:21.095 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:09:21.086+0000 7f901a51c700 -1 reset not still connected to 0x5588f8a20820 2021-03-09T03:09:21.095 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:09:21.086+0000 7f901a51c700 -1 reset not still connected to 0x5588f8a21e10 2021-03-09T03:09:21.095 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:09:21.086+0000 7f901a51c700 -1 reset not still connected to 0x5588f8cb2b10 2021-03-09T03:09:21.095 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:09:21.090+0000 7f901a51c700 -1 reset not still connected to 0x5588f8cb3810 2021-03-09T03:09:21.096 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:09:21.090+0000 7f901a51c700 -1 reset not still connected to 0x5588f96f2f20 2021-03-09T03:09:21.096 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:09:21.090+0000 7f901a51c700 -1 reset not still connected to 0x5588fa3996c0 2021-03-09T03:09:21.096 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:09:21.090+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf0d00 2021-03-09T03:09:45.267 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:09:45.261+0000 7f901a51c700 -1 reset not still connected to 0x5588f96f2be0 2021-03-09T03:10:13.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3966622254945017) > weight (0.25) 2021-03-09T03:10:13.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7892579476552541) > weight (0.25) 2021-03-09T03:10:13.307 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3048377480620821) > weight (0.25) 2021-03-09T03:10:13.307 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5157610464315762) > weight (0.25) 2021-03-09T03:10:13.307 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-03-09T03:10:15.340 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:10:15.336+0000 7f901a51c700 -1 reset not still connected to 0x5588f96f2be0 2021-03-09T03:10:46.088 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T03:10:46.089 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T03:10:46.089 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T03:10:46.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T03:10:46.086+0000 7fd8627fd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 32737) UID: 1000 2021-03-09T03:10:46.290 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:10:49.090 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T03:10:49.090 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:10:49.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:10:49.091 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T03:10:49.091 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T03:10:49.092 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T03:10:49.095 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T03:10:49.095 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T03:11:24.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6977869667644157) > weight (0.25) 2021-03-09T03:11:24.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7543412312631628) > weight (0.25) 2021-03-09T03:11:24.309 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9582128426105567) > weight (0.25) 2021-03-09T03:11:24.309 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T03:11:24.309 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T03:11:24.310 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T03:11:24.311 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T03:11:24.301+0000 7f1d6f847700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 23229) UID: 1000 2021-03-09T03:11:24.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:11:28.311 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T03:11:28.312 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:11:28.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:11:28.312 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T03:11:28.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T03:11:28.313 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T03:11:28.316 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T03:11:28.316 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T03:12:19.096 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.576468843000384) > weight (0.25) 2021-03-09T03:12:19.097 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9914746953616767) > weight (0.25) 2021-03-09T03:12:19.097 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T03:12:19.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T03:12:19.098 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T03:12:19.099 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T03:12:19.089+0000 7f0db2ff8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 26306) UID: 1000 2021-03-09T03:12:19.299 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:12:26.099 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T03:12:26.099 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:12:26.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:12:26.100 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T03:12:26.100 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T03:12:26.101 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T03:12:26.104 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T03:12:26.104 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T03:13:11.317 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6850978640883013) > weight (0.25) 2021-03-09T03:13:11.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3430400226574477) > weight (0.25) 2021-03-09T03:13:11.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3755566385707757) > weight (0.25) 2021-03-09T03:13:11.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8119796044109817) > weight (0.25) 2021-03-09T03:13:11.319 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T03:13:36.105 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9762201981034258) > weight (0.25) 2021-03-09T03:13:36.106 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7999739942622272) > weight (0.25) 2021-03-09T03:13:36.106 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T03:13:36.107 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T03:13:36.108 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T03:13:36.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T03:13:36.103+0000 7f9e38cf3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 33020) UID: 1000 2021-03-09T03:13:36.309 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:13:44.109 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T03:13:44.109 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:13:44.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:13:44.110 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T03:13:44.110 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T03:13:44.110 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T03:13:44.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T03:13:44.114 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T03:14:07.339 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:14:07.334+0000 7f901a51c700 -1 reset not still connected to 0x5588f96f2be0 2021-03-09T03:15:07.320 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2729229706655645) > weight (0.25) 2021-03-09T03:15:07.321 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45348698406824395) > weight (0.25) 2021-03-09T03:15:07.322 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8884066847906076) > weight (0.25) 2021-03-09T03:15:07.322 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T03:15:07.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T03:15:07.324 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T03:15:07.325 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T03:15:07.318+0000 7fea5644b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 32920) UID: 1000 2021-03-09T03:15:07.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:15:11.325 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T03:15:11.326 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:15:11.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:15:11.327 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T03:15:11.327 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T03:15:11.327 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T03:15:11.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T03:15:11.330 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-03-09T03:15:12.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3003954927970207) > weight (0.25) 2021-03-09T03:15:12.115 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7329193040990547) > weight (0.25) 2021-03-09T03:15:12.115 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T03:15:12.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T03:15:12.116 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T03:15:12.117 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T03:15:12.110+0000 7f8883f97700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 33112) UID: 1000 2021-03-09T03:15:12.317 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:15:15.117 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T03:15:15.117 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:15:15.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:15:15.118 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T03:15:15.118 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T03:15:15.118 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T03:15:15.121 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T03:15:15.122 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T03:16:30.331 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4648234508388419) > weight (0.25) 2021-03-09T03:16:30.332 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T03:16:30.333 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T03:16:30.333 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T03:16:30.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T03:16:30.329+0000 7f44b5c31700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 30228) UID: 1000 2021-03-09T03:16:30.534 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:16:39.334 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T03:16:39.335 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:16:39.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:16:39.335 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T03:16:39.336 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T03:16:39.336 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T03:16:39.339 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T03:16:39.340 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T03:17:08.123 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.41767085333057263) > weight (0.25) 2021-03-09T03:17:08.123 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25243048650505717) > weight (0.25) 2021-03-09T03:17:08.124 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7030711088688846) > weight (0.25) 2021-03-09T03:17:08.124 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5744135443896965) > weight (0.25) 2021-03-09T03:17:08.124 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T03:18:04.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.40476896378766614) > weight (0.25) 2021-03-09T03:18:04.341 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5646828307258698) > weight (0.25) 2021-03-09T03:18:04.342 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6576277121701323) > weight (0.25) 2021-03-09T03:18:04.342 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5429492022196003) > weight (0.25) 2021-03-09T03:18:04.342 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T03:18:57.125 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8713440318809548) > weight (0.25) 2021-03-09T03:18:57.127 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8059047042472834) > weight (0.25) 2021-03-09T03:18:57.127 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.32264268843839194) > weight (0.25) 2021-03-09T03:18:57.127 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T03:18:57.129 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T03:18:57.129 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T03:18:57.130 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T03:18:57.126+0000 7f2521621700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 28874) UID: 1000 2021-03-09T03:18:57.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:19:01.131 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T03:19:01.131 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:19:01.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:19:01.132 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T03:19:01.132 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T03:19:01.133 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T03:19:01.135 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T03:19:01.136 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T03:19:10.343 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T03:19:10.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T03:19:10.344 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T03:19:10.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T03:19:10.342+0000 7f23bad00700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 30861) UID: 1000 2021-03-09T03:19:10.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:19:12.346 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T03:19:12.346 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:19:12.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:19:12.347 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T03:19:12.347 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T03:19:12.347 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T03:19:12.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T03:19:12.351 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T03:20:08.136 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.28718024890628857) > weight (0.25) 2021-03-09T03:20:08.137 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9167386632295439) > weight (0.25) 2021-03-09T03:20:08.137 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T03:20:08.138 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T03:20:08.138 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T03:20:08.138 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T03:20:08.139 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:20:08.139 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T03:20:08.134+0000 7f273ec5f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 33298) UID: 1000 2021-03-09T03:20:08.339 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:20:14.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:20:14.241 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T03:20:14.241 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T03:20:14.241 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T03:20:14.244 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T03:20:14.244 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T03:20:32.352 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7831615808091158) > weight (0.25) 2021-03-09T03:20:32.352 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5011710454839405) > weight (0.25) 2021-03-09T03:20:32.353 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3068456379800196) > weight (0.25) 2021-03-09T03:20:32.353 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T03:20:32.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T03:20:32.354 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T03:20:32.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T03:20:32.349+0000 7fe14276c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 33207) UID: 1000 2021-03-09T03:20:32.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:20:35.354 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T03:20:35.355 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:20:35.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:20:35.355 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T03:20:35.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T03:20:35.356 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T03:20:35.358 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T03:20:35.359 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T03:21:46.245 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T03:21:46.246 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T03:21:46.247 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T03:21:46.248 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T03:21:46.244+0000 7f3cd4cdf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 32829) UID: 1000 2021-03-09T03:21:46.448 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:21:48.247 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T03:21:48.248 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:21:48.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:21:48.248 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T03:21:48.249 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T03:21:48.249 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T03:21:48.252 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T03:21:48.252 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T03:22:18.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7981335517285639) > weight (0.25) 2021-03-09T03:22:18.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6812928514172931) > weight (0.25) 2021-03-09T03:22:18.361 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T03:22:18.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T03:22:18.361 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T03:22:18.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T03:22:18.360+0000 7f018bae3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 26820) UID: 1000 2021-03-09T03:22:18.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:22:25.363 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T03:22:25.363 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:22:25.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:22:25.364 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T03:22:25.364 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T03:22:25.364 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T03:22:25.367 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T03:22:25.367 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T03:22:52.253 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.34866247824210217) > weight (0.25) 2021-03-09T03:22:52.253 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T03:22:52.254 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T03:22:52.254 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T03:22:52.255 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T03:22:52.251+0000 7ff8ddfa4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 31140) UID: 1000 2021-03-09T03:22:52.456 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:22:56.255 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T03:22:56.255 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:22:56.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:22:56.256 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T03:22:56.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T03:22:56.256 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T03:22:56.259 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T03:22:56.260 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T03:24:02.369 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7903432715425989) > weight (0.25) 2021-03-09T03:24:02.369 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T03:24:02.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T03:24:02.370 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T03:24:02.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T03:24:02.366+0000 7f81cc2a4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 33397) UID: 1000 2021-03-09T03:24:02.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:24:07.371 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T03:24:07.371 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:24:07.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:24:07.372 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T03:24:07.373 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T03:24:07.373 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T03:24:07.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T03:24:07.388 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T03:24:44.260 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7892231434992388) > weight (0.25) 2021-03-09T03:24:44.261 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T03:24:44.262 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T03:24:44.262 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T03:24:44.263 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T03:24:44.261+0000 7fbd9b9e3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 34073) UID: 1000 2021-03-09T03:24:44.463 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:24:45.263 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T03:24:45.263 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:24:45.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:24:45.264 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T03:24:45.264 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T03:24:45.265 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T03:24:45.267 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T03:24:45.267 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T03:25:27.389 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9362088094012772) > weight (0.25) 2021-03-09T03:25:27.390 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7080783504343189) > weight (0.25) 2021-03-09T03:25:27.390 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.47029943209313985) > weight (0.25) 2021-03-09T03:25:27.390 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8964549414576288) > weight (0.25) 2021-03-09T03:25:27.391 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T03:26:07.269 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.45216112741734793) > weight (0.25) 2021-03-09T03:26:07.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5386372947012276) > weight (0.25) 2021-03-09T03:26:07.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8313382199970387) > weight (0.25) 2021-03-09T03:26:07.270 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T03:26:07.271 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T03:26:07.271 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T03:26:07.272 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T03:26:07.268+0000 7f1c2749b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 33502) UID: 1000 2021-03-09T03:26:07.473 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:26:08.272 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T03:26:08.273 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:26:08.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:26:08.274 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T03:26:08.274 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T03:26:08.274 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T03:26:08.277 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T03:26:08.277 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T03:26:36.392 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5956765889627392) > weight (0.25) 2021-03-09T03:26:36.393 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7883715326807665) > weight (0.25) 2021-03-09T03:26:36.393 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5142835774806375) > weight (0.25) 2021-03-09T03:26:36.393 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9049603495625582) > weight (0.25) 2021-03-09T03:26:36.393 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T03:27:14.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.310326229777253) > weight (0.25) 2021-03-09T03:27:14.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7580156098497091) > weight (0.25) 2021-03-09T03:27:14.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.615372357250497) > weight (0.25) 2021-03-09T03:27:14.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3073209755283859) > weight (0.25) 2021-03-09T03:27:14.280 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T03:28:24.394 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.746593317092595) > weight (0.25) 2021-03-09T03:28:24.395 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T03:28:24.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T03:28:24.397 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T03:28:24.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T03:28:24.394+0000 7fc41f822700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 34164) UID: 1000 2021-03-09T03:28:24.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:28:28.397 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T03:28:28.398 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:28:28.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:28:28.400 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T03:28:28.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T03:28:28.401 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T03:28:28.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T03:28:28.406 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T03:28:38.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7419256768125083) > weight (0.25) 2021-03-09T03:28:38.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9868434042062857) > weight (0.25) 2021-03-09T03:28:38.282 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T03:28:38.283 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T03:28:38.284 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T03:28:38.284 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T03:28:38.285 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:28:38.286 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T03:28:38.277+0000 7f0525ca2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 33692) UID: 1000 2021-03-09T03:28:38.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:28:44.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:28:44.388 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T03:28:44.388 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T03:28:44.389 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T03:28:44.393 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T03:28:44.393 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T03:30:17.407 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2847902251570239) > weight (0.25) 2021-03-09T03:30:17.408 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T03:30:17.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T03:30:17.410 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T03:30:17.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T03:30:17.408+0000 7f34e61de700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 34450) UID: 1000 2021-03-09T03:30:17.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:30:23.394 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.48329109506301027) > weight (0.25) 2021-03-09T03:30:23.394 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.38238308158556356) > weight (0.25) 2021-03-09T03:30:23.395 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8655724001030323) > weight (0.25) 2021-03-09T03:30:23.395 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T03:30:23.395 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T03:30:23.396 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T03:30:23.396 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T03:30:23.396 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:30:23.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T03:30:23.392+0000 7fcb91836700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 34359) UID: 1000 2021-03-09T03:30:23.411 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T03:30:23.412 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:30:23.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:30:23.413 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T03:30:23.413 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T03:30:23.413 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T03:30:23.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T03:30:23.416 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T03:30:23.598 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:30:29.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:30:29.499 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T03:30:29.500 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T03:30:29.500 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T03:30:29.503 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T03:30:29.503 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T03:31:33.504 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.379443440466566) > weight (0.25) 2021-03-09T03:31:33.505 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T03:31:33.506 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T03:31:33.506 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T03:31:33.507 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T03:31:33.502+0000 7f188d0a7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 34264) UID: 1000 2021-03-09T03:31:33.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:31:42.507 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T03:31:42.508 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:31:42.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:31:42.509 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T03:31:42.509 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T03:31:42.509 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T03:31:42.512 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T03:31:42.513 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-03-09T03:32:14.604 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T03:32:14.590+0000 7fc0e0d45700 -1 reset not still connected to 0x55e8d5746080 2021-03-09T03:32:18.417 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7206951065917354) > weight (0.25) 2021-03-09T03:32:18.418 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8719640660402489) > weight (0.25) 2021-03-09T03:32:18.418 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T03:32:18.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T03:32:18.419 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T03:32:18.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T03:32:18.414+0000 7f01f9538700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 33974) UID: 1000 2021-03-09T03:32:18.620 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:32:19.420 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T03:32:19.420 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:32:19.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:32:19.421 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T03:32:19.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T03:32:19.422 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T03:32:19.424 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T03:32:19.425 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-03-09T03:33:00.514 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7518060539491672) > weight (0.25) 2021-03-09T03:33:00.514 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8995191361775982) > weight (0.25) 2021-03-09T03:33:00.515 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9125338175002258) > weight (0.25) 2021-03-09T03:33:00.515 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8578107528143875) > weight (0.25) 2021-03-09T03:33:00.515 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T03:33:34.426 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T03:33:34.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T03:33:34.428 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T03:33:34.428 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T03:33:34.421+0000 7fe4904d9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 33593) UID: 1000 2021-03-09T03:33:34.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:33:35.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T03:33:35.581+0000 7f4897004700 -1 rbd::mirror::ImageReplayer: 0x55c591a30780 [3/08a00e86-016b-46cb-bca2-dcf2f528e189] start: already running 2021-03-09T03:33:35.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T03:33:35.581+0000 7f4897004700 -1 rbd::mirror::ImageReplayer: 0x55c58d9fd200 [3/1957e71d-23c8-464e-a815-0768c7c29be6] start: already running 2021-03-09T03:33:35.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T03:33:35.585+0000 7f4896803700 -1 rbd::mirror::ImageReplayer: 0x55c58d9fdc00 [3/76a412aa-913f-4f54-ad5f-0dbad3c607cd] start: already running 2021-03-09T03:33:35.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T03:33:35.585+0000 7f4897004700 -1 rbd::mirror::ImageReplayer: 0x55c58d9fc800 [3/8853c04a-4df4-47fb-9df9-6cebf19cc680] start: already running 2021-03-09T03:33:35.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T03:33:35.585+0000 7f4897004700 -1 rbd::mirror::ImageReplayer: 0x55c591a30f00 [3/90629254-6fb9-41d9-b690-e94fb222ffa6] start: already running 2021-03-09T03:33:35.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T03:33:35.585+0000 7f4896803700 -1 rbd::mirror::ImageReplayer: 0x55c591a33200 [3/aec0a6af-f0ca-401c-a5d1-8f250674262a] start: already running 2021-03-09T03:33:42.429 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T03:33:42.429 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:33:42.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:33:42.430 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T03:33:42.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T03:33:42.431 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T03:33:42.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T03:33:42.435 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T03:34:59.517 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6057307245679697) > weight (0.25) 2021-03-09T03:34:59.517 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T03:34:59.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T03:34:59.519 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T03:34:59.520 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T03:34:59.515+0000 7fe29f88c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 34834) UID: 1000 2021-03-09T03:34:59.721 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:35:01.520 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T03:35:01.521 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:35:01.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:35:01.522 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T03:35:01.522 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T03:35:01.522 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T03:35:01.525 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T03:35:01.526 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T03:35:41.436 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5639929432416577) > weight (0.25) 2021-03-09T03:35:41.436 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4278018118275263) > weight (0.25) 2021-03-09T03:35:41.437 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7050623161654652) > weight (0.25) 2021-03-09T03:35:41.437 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4756871278442205) > weight (0.25) 2021-03-09T03:35:41.437 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T03:35:44.616 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:35:44.612+0000 7f901a51c700 -1 reset not still connected to 0x5588fa399520 2021-03-09T03:35:48.506 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:35:48.504+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf2700 2021-03-09T03:36:06.527 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38251599676704595) > weight (0.25) 2021-03-09T03:36:06.527 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.46098144356805526) > weight (0.25) 2021-03-09T03:36:06.528 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.35003584098514495) > weight (0.25) 2021-03-09T03:36:06.528 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8006293793054786) > weight (0.25) 2021-03-09T03:36:06.528 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T03:36:46.438 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.509581858758472) > weight (0.25) 2021-03-09T03:36:46.439 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7271502169968866) > weight (0.25) 2021-03-09T03:36:46.439 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5548477772643323) > weight (0.25) 2021-03-09T03:36:46.439 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5222316649858117) > weight (0.25) 2021-03-09T03:36:46.439 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T03:36:49.608 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:36:49.601+0000 7f082a074700 -1 reset not still connected to 0x55a9911249c0 2021-03-09T03:36:49.609 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:36:49.601+0000 7f082a074700 -1 reset not still connected to 0x55a9968aa220 2021-03-09T03:36:49.612 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:36:49.605+0000 7f082a074700 -1 reset not still connected to 0x55a9975a1a00 2021-03-09T03:36:49.613 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:36:49.605+0000 7f082a074700 -1 reset not still connected to 0x55a9975a1fb0 2021-03-09T03:36:49.613 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:36:49.605+0000 7f082a074700 -1 reset not still connected to 0x55a9f04e8dd0 2021-03-09T03:36:49.614 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:36:49.609+0000 7f082a074700 -1 reset not still connected to 0x55a9f04e92b0 2021-03-09T03:37:37.529 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5461607943384538) > weight (0.25) 2021-03-09T03:37:37.530 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9404685005686001) > weight (0.25) 2021-03-09T03:37:37.531 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T03:37:37.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T03:37:37.532 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T03:37:37.533 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T03:37:37.529+0000 7f190d8ca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 34549) UID: 1000 2021-03-09T03:37:37.733 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:37:46.533 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T03:37:46.534 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:37:46.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:37:46.535 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T03:37:46.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T03:37:46.535 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T03:37:46.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T03:37:46.539 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-03-09T03:38:18.440 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3943127700791743) > weight (0.25) 2021-03-09T03:38:18.441 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T03:38:18.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T03:38:18.443 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T03:38:18.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T03:38:18.440+0000 7f213f688700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 34642) UID: 1000 2021-03-09T03:38:18.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:38:26.444 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T03:38:26.444 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:38:26.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:38:26.445 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T03:38:26.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T03:38:26.446 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T03:38:26.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T03:38:26.449 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T03:38:57.540 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5865978007393559) > weight (0.25) 2021-03-09T03:38:57.541 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9051315265876413) > weight (0.25) 2021-03-09T03:38:57.541 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T03:38:57.542 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T03:38:57.542 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T03:38:57.543 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T03:38:57.539+0000 7f5b2ec48700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 35221) UID: 1000 2021-03-09T03:38:57.744 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:39:02.543 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T03:39:02.544 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:39:02.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:39:02.545 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T03:39:02.545 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T03:39:02.545 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T03:39:02.548 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T03:39:02.549 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T03:40:07.450 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5012198221997971) > weight (0.25) 2021-03-09T03:40:07.451 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3015690250086325) > weight (0.25) 2021-03-09T03:40:07.451 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T03:40:07.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T03:40:07.453 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T03:40:07.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T03:40:07.450+0000 7f6df086b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 34925) UID: 1000 2021-03-09T03:40:07.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:40:09.453 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T03:40:09.454 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:40:09.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:40:09.454 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T03:40:09.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T03:40:09.455 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T03:40:09.458 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T03:40:09.458 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T03:40:14.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7253082058848064) > weight (0.25) 2021-03-09T03:40:14.551 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T03:40:14.551 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T03:40:14.551 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T03:40:14.552 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T03:40:14.550+0000 7f282d2fc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 35128) UID: 1000 2021-03-09T03:40:14.754 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:40:17.553 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T03:40:17.553 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:40:17.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:40:17.554 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T03:40:17.554 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T03:40:17.555 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T03:40:17.558 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T03:40:17.558 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T03:41:59.559 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T03:41:59.562 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T03:41:59.563 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T03:41:59.565 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T03:41:59.560+0000 7f1bb457a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 33883) UID: 1000 2021-03-09T03:41:59.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:42:06.459 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9971607799527888) > weight (0.25) 2021-03-09T03:42:06.459 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3108936163490056) > weight (0.25) 2021-03-09T03:42:06.460 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T03:42:06.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T03:42:06.461 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T03:42:06.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T03:42:06.460+0000 7f779307b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 35504) UID: 1000 2021-03-09T03:42:06.565 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T03:42:06.565 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:42:06.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:42:06.566 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T03:42:06.566 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T03:42:06.567 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T03:42:06.570 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T03:42:06.570 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T03:42:06.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:42:09.461 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T03:42:09.462 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:42:09.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:42:09.462 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T03:42:09.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T03:42:09.463 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T03:42:09.466 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T03:42:09.466 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T03:42:32.791 INFO:tasks.ceph.cluster2.osd.0.smithi144.stderr:2021-03-09T03:42:32.787+0000 7f0e42124700 -1 reset not still connected to 0x55740d8db400 2021-03-09T03:43:08.572 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9283955305532766) > weight (0.25) 2021-03-09T03:43:08.572 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T03:43:08.573 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T03:43:08.573 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T03:43:08.574 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T03:43:08.571+0000 7fc11ee63700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 35603) UID: 1000 2021-03-09T03:43:08.775 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:43:13.575 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T03:43:13.575 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:43:13.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:43:13.576 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T03:43:13.576 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T03:43:13.577 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T03:43:13.579 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T03:43:13.579 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T03:43:15.467 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T03:43:15.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T03:43:15.468 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T03:43:15.469 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T03:43:15.467+0000 7fd524756700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 35024) UID: 1000 2021-03-09T03:43:15.669 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:43:16.845 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:43:16.839+0000 7f901a51c700 -1 reset not still connected to 0x5588f277fdc0 2021-03-09T03:43:16.845 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:43:16.839+0000 7f901a51c700 -1 reset not still connected to 0x5588fa3996c0 2021-03-09T03:43:16.845 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:43:16.839+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf0f70 2021-03-09T03:43:21.469 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T03:43:21.470 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:43:21.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:43:21.470 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T03:43:21.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T03:43:21.471 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T03:43:21.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T03:43:21.477 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T03:44:16.581 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5067241413625808) > weight (0.25) 2021-03-09T03:44:16.582 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7514675377073069) > weight (0.25) 2021-03-09T03:44:16.582 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.914654232365084) > weight (0.25) 2021-03-09T03:44:16.582 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3351527399141223) > weight (0.25) 2021-03-09T03:44:16.582 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T03:44:32.773 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:44:32.765+0000 7f082a074700 -1 reset not still connected to 0x55a9f121d4d0 2021-03-09T03:44:43.479 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.46545455897704024) > weight (0.25) 2021-03-09T03:44:43.480 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.28079709762598215) > weight (0.25) 2021-03-09T03:44:43.480 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.615606363240288) > weight (0.25) 2021-03-09T03:44:43.480 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7876386808667066) > weight (0.25) 2021-03-09T03:44:43.480 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T03:45:46.584 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T03:45:46.585 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T03:45:46.586 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T03:45:46.587 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T03:45:46.584+0000 7f47c3c9e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 35695) UID: 1000 2021-03-09T03:45:46.787 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:45:54.587 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T03:45:54.588 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:45:54.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:45:54.589 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T03:45:54.589 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T03:45:54.590 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T03:45:54.593 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T03:45:54.593 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T03:46:29.482 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T03:46:29.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T03:46:29.483 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T03:46:29.483 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T03:46:29.483 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:46:29.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T03:46:29.479+0000 7efcc39db700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 35977) UID: 1000 2021-03-09T03:46:29.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:46:35.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:46:35.586 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T03:46:35.586 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T03:46:35.586 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T03:46:35.589 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T03:46:35.589 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T03:47:08.594 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3153049470533148) > weight (0.25) 2021-03-09T03:47:08.595 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9965156918627711) > weight (0.25) 2021-03-09T03:47:08.595 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.32490996311227527) > weight (0.25) 2021-03-09T03:47:08.596 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.36293289918437377) > weight (0.25) 2021-03-09T03:47:08.596 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T03:48:21.590 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T03:48:21.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T03:48:21.592 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T03:48:21.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T03:48:21.589+0000 7fac1663e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 36249) UID: 1000 2021-03-09T03:48:21.793 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:48:26.593 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T03:48:26.593 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:48:26.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:48:26.594 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T03:48:26.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T03:48:26.595 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T03:48:26.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T03:48:26.598 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T03:48:56.597 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T03:48:56.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T03:48:56.599 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T03:48:56.600 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T03:48:56.596+0000 7fdd3e801700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 36158) UID: 1000 2021-03-09T03:48:56.800 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:49:02.600 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T03:49:02.601 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:49:02.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:49:02.601 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T03:49:02.602 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T03:49:02.602 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T03:49:02.605 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T03:49:02.605 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T03:49:13.790 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T03:49:13.788+0000 7fc0e0d45700 -1 reset not still connected to 0x55e9194fde10 2021-03-09T03:49:32.599 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T03:49:32.600 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T03:49:32.601 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T03:49:32.601 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T03:49:32.600+0000 7f2b0aef6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 36351) UID: 1000 2021-03-09T03:49:32.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:49:37.602 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T03:49:37.602 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:49:37.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:49:37.603 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T03:49:37.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T03:49:37.604 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T03:49:37.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T03:49:37.607 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T03:49:41.947 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:49:41.944+0000 7f082a074700 -1 reset not still connected to 0x55a9975a3b50 2021-03-09T03:49:41.948 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:49:41.944+0000 7f082a074700 -1 reset not still connected to 0x55aa12b1e4e0 2021-03-09T03:49:43.798 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:49:43.796+0000 7f901a51c700 -1 reset not still connected to 0x558904ab30c0 2021-03-09T03:50:03.956 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:50:03.951+0000 7f082a074700 -1 reset not still connected to 0x55a9975a2a40 2021-03-09T03:50:03.957 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:50:03.951+0000 7f082a074700 -1 reset not still connected to 0x55a9975a30c0 2021-03-09T03:50:03.957 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:50:03.951+0000 7f082a074700 -1 reset not still connected to 0x55a9975a39b0 2021-03-09T03:50:03.958 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T03:50:03.951+0000 7f082a074700 -1 reset not still connected to 0x55a9ad1b6270 2021-03-09T03:50:59.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8446413089372286) > weight (0.25) 2021-03-09T03:50:59.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5341723680742175) > weight (0.25) 2021-03-09T03:50:59.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8918926370672707) > weight (0.25) 2021-03-09T03:50:59.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5284536921005115) > weight (0.25) 2021-03-09T03:50:59.608 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T03:51:20.608 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T03:51:20.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T03:51:20.611 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T03:51:20.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T03:51:20.606+0000 7feb92ea4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 36541) UID: 1000 2021-03-09T03:51:20.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:51:29.612 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T03:51:29.613 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:51:29.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:51:29.614 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T03:51:29.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T03:51:29.614 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T03:51:29.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T03:51:29.617 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T03:52:05.609 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.39385615239790406) > weight (0.25) 2021-03-09T03:52:05.609 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.863390361420347) > weight (0.25) 2021-03-09T03:52:05.610 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T03:52:05.610 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T03:52:05.611 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T03:52:05.612 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T03:52:05.605+0000 7f199f9f3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 35411) UID: 1000 2021-03-09T03:52:05.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:52:08.612 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T03:52:08.612 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:52:08.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:52:08.613 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T03:52:08.613 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T03:52:08.614 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T03:52:08.617 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T03:52:08.617 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T03:53:20.619 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3558269518792564) > weight (0.25) 2021-03-09T03:53:20.620 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8472594118041458) > weight (0.25) 2021-03-09T03:53:20.620 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.250336385872002) > weight (0.25) 2021-03-09T03:53:20.620 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T03:53:20.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T03:53:20.622 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T03:53:20.623 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T03:53:20.620+0000 7f4897805700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 33783) UID: 1000 2021-03-09T03:53:20.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:53:21.991 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:53:21.990+0000 7f901a51c700 -1 reset not still connected to 0x5588f277fdc0 2021-03-09T03:53:21.991 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:53:21.990+0000 7f901a51c700 -1 reset not still connected to 0x5588f8a201a0 2021-03-09T03:53:21.991 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:53:21.990+0000 7f901a51c700 -1 reset not still connected to 0x5588fa3996c0 2021-03-09T03:53:22.012 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T03:53:22.006+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad07485b0 2021-03-09T03:53:29.623 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T03:53:29.624 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:53:29.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:53:29.624 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T03:53:29.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T03:53:29.625 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T03:53:29.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T03:53:29.629 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T03:53:33.619 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T03:53:33.619 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T03:53:33.620 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T03:53:33.621 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T03:53:33.620+0000 7f12e09c4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 36450) UID: 1000 2021-03-09T03:53:33.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:53:35.621 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T03:53:35.621 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:53:35.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:53:35.622 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T03:53:35.622 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T03:53:35.623 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T03:53:35.625 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T03:53:35.626 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T03:54:49.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8721828585339273) > weight (0.25) 2021-03-09T03:54:49.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.43985458021371104) > weight (0.25) 2021-03-09T03:54:49.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9782729198212775) > weight (0.25) 2021-03-09T03:54:49.628 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6826735785147988) > weight (0.25) 2021-03-09T03:54:49.628 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T03:55:14.631 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8005317150470745) > weight (0.25) 2021-03-09T03:55:14.631 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.31454155913751825) > weight (0.25) 2021-03-09T03:55:14.632 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T03:55:14.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T03:55:14.633 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T03:55:14.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T03:55:14.630+0000 7fe260616700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 35786) UID: 1000 2021-03-09T03:55:14.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:55:18.634 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T03:55:18.634 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:55:18.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:55:18.635 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T03:55:18.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T03:55:18.636 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T03:55:18.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T03:55:18.639 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T03:56:19.640 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4002745942756015) > weight (0.25) 2021-03-09T03:56:19.641 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7551624283350463) > weight (0.25) 2021-03-09T03:56:19.641 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4943916514522536) > weight (0.25) 2021-03-09T03:56:19.642 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T03:56:19.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T03:56:19.643 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T03:56:19.644 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T03:56:19.641+0000 7f4798848700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 36834) UID: 1000 2021-03-09T03:56:19.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:56:25.644 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T03:56:25.644 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:56:25.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:56:25.645 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T03:56:25.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T03:56:25.646 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T03:56:25.648 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T03:56:25.649 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T03:56:28.629 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8004245164614685) > weight (0.25) 2021-03-09T03:56:28.629 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7450103952123993) > weight (0.25) 2021-03-09T03:56:28.629 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.40551215257957185) > weight (0.25) 2021-03-09T03:56:28.630 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T03:56:28.630 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T03:56:28.630 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T03:56:28.631 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T03:56:28.631 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:56:28.632 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T03:56:28.628+0000 7fde06f39700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 34741) UID: 1000 2021-03-09T03:56:28.832 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:56:34.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:56:34.734 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T03:56:34.734 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T03:56:34.734 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T03:56:34.737 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T03:56:34.737 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T03:57:39.050 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:57:39.046+0000 7f901a51c700 -1 reset not still connected to 0x558904ab1c70 2021-03-09T03:57:39.739 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2818898489074039) > weight (0.25) 2021-03-09T03:57:39.739 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8039736939677514) > weight (0.25) 2021-03-09T03:57:39.739 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.31090351018941187) > weight (0.25) 2021-03-09T03:57:39.740 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6087620964347231) > weight (0.25) 2021-03-09T03:57:39.740 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T03:58:05.650 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9123840829172102) > weight (0.25) 2021-03-09T03:58:05.651 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3695001137283841) > weight (0.25) 2021-03-09T03:58:05.652 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8929414667221591) > weight (0.25) 2021-03-09T03:58:05.652 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.46679077097902255) > weight (0.25) 2021-03-09T03:58:05.652 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T03:59:03.741 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T03:59:03.745 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T03:59:03.745 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T03:59:03.746 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T03:59:03.746 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:59:03.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T03:59:03.742+0000 7f923cca7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 36933) UID: 1000 2021-03-09T03:59:03.946 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:59:04.887 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:59:04.884+0000 7f901a51c700 -1 reset not still connected to 0x5588f8a22d80 2021-03-09T03:59:04.888 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T03:59:04.884+0000 7f901a51c700 -1 reset not still connected to 0x558904ab30c0 2021-03-09T03:59:09.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:59:09.851 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T03:59:09.852 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T03:59:09.852 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T03:59:09.856 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T03:59:09.857 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T03:59:11.653 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6038355688989037) > weight (0.25) 2021-03-09T03:59:11.659 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3314151086462743) > weight (0.25) 2021-03-09T03:59:11.659 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T03:59:11.660 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T03:59:11.661 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T03:59:11.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T03:59:11.658+0000 7fa5c2b45700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 37027) UID: 1000 2021-03-09T03:59:11.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T03:59:17.662 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T03:59:17.662 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T03:59:17.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T03:59:17.663 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T03:59:17.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T03:59:17.664 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T03:59:17.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T03:59:17.667 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T04:00:24.668 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8152086704580126) > weight (0.25) 2021-03-09T04:00:24.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7997155463265125) > weight (0.25) 2021-03-09T04:00:24.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9534365400954526) > weight (0.25) 2021-03-09T04:00:24.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9348227077404268) > weight (0.25) 2021-03-09T04:00:24.669 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T04:01:08.858 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T04:01:08.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T04:01:08.861 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T04:01:08.862 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:01:08.859+0000 7fe39bb05700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 37320) UID: 1000 2021-03-09T04:01:09.062 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:01:14.862 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T04:01:14.863 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:01:14.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:01:14.864 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T04:01:14.864 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T04:01:14.865 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T04:01:14.868 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T04:01:14.868 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T04:01:24.671 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6439854167842292) > weight (0.25) 2021-03-09T04:01:24.673 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.559965389577109) > weight (0.25) 2021-03-09T04:01:24.673 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7010114646311991) > weight (0.25) 2021-03-09T04:01:24.674 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6460181437971888) > weight (0.25) 2021-03-09T04:01:24.674 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T04:02:48.870 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T04:02:48.876 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T04:02:48.876 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T04:02:48.877 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:02:48.870+0000 7f81abf04700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 37511) UID: 1000 2021-03-09T04:02:49.077 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:02:53.877 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T04:02:53.878 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:02:53.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:02:53.879 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T04:02:53.879 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T04:02:53.880 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T04:02:53.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T04:02:53.883 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T04:03:06.675 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8663977138953572) > weight (0.25) 2021-03-09T04:03:06.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5996693559342228) > weight (0.25) 2021-03-09T04:03:06.676 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T04:03:06.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T04:03:06.677 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T04:03:06.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T04:03:06.674+0000 7f29261af700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 37411) UID: 1000 2021-03-09T04:03:06.878 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:03:08.678 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T04:03:08.678 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:03:08.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:03:08.679 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T04:03:08.679 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T04:03:08.680 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T04:03:08.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T04:03:08.683 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-03-09T04:03:39.296 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:03:39.292+0000 7f901a51c700 -1 reset not still connected to 0x5588f277fdc0 2021-03-09T04:03:39.297 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:03:39.292+0000 7f901a51c700 -1 reset not still connected to 0x5588f8a201a0 2021-03-09T04:03:39.297 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:03:39.292+0000 7f901a51c700 -1 reset not still connected to 0x5588f8a230c0 2021-03-09T04:03:39.297 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:03:39.292+0000 7f901a51c700 -1 reset not still connected to 0x5588f96f35a0 2021-03-09T04:03:39.297 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:03:39.292+0000 7f901a51c700 -1 reset not still connected to 0x5588fa3996c0 2021-03-09T04:04:15.885 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T04:04:15.886 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T04:04:15.886 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T04:04:15.887 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:04:15.884+0000 7f9e8c633700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 37604) UID: 1000 2021-03-09T04:04:16.087 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:04:16.888 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T04:04:16.888 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:04:16.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:04:16.889 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T04:04:16.889 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T04:04:16.890 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T04:04:16.892 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T04:04:16.893 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T04:04:29.683 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T04:04:29.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T04:04:29.685 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T04:04:29.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:04:29.684+0000 7f3bd1d1d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 36641) UID: 1000 2021-03-09T04:04:29.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:04:33.685 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T04:04:33.686 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:04:33.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:04:33.686 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T04:04:33.687 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T04:04:33.687 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T04:04:33.690 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T04:04:33.690 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T04:06:15.894 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8604778301276594) > weight (0.25) 2021-03-09T04:06:15.895 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:06:15.895 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:06:15.896 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T04:06:15.897 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:06:15.894+0000 7f561b3da700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 35886) UID: 1000 2021-03-09T04:06:16.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:06:17.099 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T04:06:17.094+0000 7f082a074700 -1 reset not still connected to 0x55a98f9cd110 2021-03-09T04:06:17.100 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T04:06:17.094+0000 7f082a074700 -1 reset not still connected to 0x55a991ffd520 2021-03-09T04:06:17.897 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:06:17.898 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:06:17.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:06:17.899 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:06:17.899 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:06:17.899 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:06:17.902 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:06:17.903 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T04:06:21.691 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T04:06:21.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T04:06:21.692 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T04:06:21.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:06:21.690+0000 7f579b57a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 37886) UID: 1000 2021-03-09T04:06:21.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:06:25.694 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T04:06:25.694 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:06:25.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:06:25.695 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T04:06:25.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T04:06:25.695 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T04:06:25.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T04:06:25.699 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T04:08:02.904 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8316314887544428) > weight (0.25) 2021-03-09T04:08:02.905 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:08:02.906 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:08:02.906 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T04:08:02.907 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:08:02.905+0000 7f42d4d6c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 37988) UID: 1000 2021-03-09T04:08:03.107 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:08:04.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7402588884801165) > weight (0.25) 2021-03-09T04:08:04.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.311867394174433) > weight (0.25) 2021-03-09T04:08:04.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6188957087898102) > weight (0.25) 2021-03-09T04:08:04.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.45439419082177046) > weight (0.25) 2021-03-09T04:08:04.700 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T04:08:10.907 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:08:10.907 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:08:10.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:08:10.908 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:08:10.908 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:08:10.909 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:08:10.911 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:08:10.912 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T04:09:20.701 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7829912808584224) > weight (0.25) 2021-03-09T04:09:20.702 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9033401320572868) > weight (0.25) 2021-03-09T04:09:20.702 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4689826197769056) > weight (0.25) 2021-03-09T04:09:20.702 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T04:09:20.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T04:09:20.704 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T04:09:20.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T04:09:20.699+0000 7f7dd0104700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 37126) UID: 1000 2021-03-09T04:09:20.905 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:09:27.913 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5940517524781218) > weight (0.25) 2021-03-09T04:09:27.913 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:09:27.914 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:09:27.914 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T04:09:27.915 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:09:27.915 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:09:27.916 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:09:27.911+0000 7fcbb2eff700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 38180) UID: 1000 2021-03-09T04:09:28.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:09:28.705 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T04:09:28.705 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:09:28.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:09:28.706 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T04:09:28.706 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T04:09:28.707 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T04:09:28.710 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T04:09:28.710 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-03-09T04:09:34.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:09:34.018 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:09:34.019 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:09:34.019 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:09:34.021 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:09:34.022 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T04:10:47.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6894339201808277) > weight (0.25) 2021-03-09T04:10:47.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5101266566098706) > weight (0.25) 2021-03-09T04:10:47.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.714969300685109) > weight (0.25) 2021-03-09T04:10:47.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4033737729232686) > weight (0.25) 2021-03-09T04:10:47.712 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T04:10:48.023 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.32712557062857217) > weight (0.25) 2021-03-09T04:10:48.023 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:10:48.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:10:48.025 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T04:10:48.026 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:10:48.022+0000 7f2067f63700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 38363) UID: 1000 2021-03-09T04:10:48.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:10:49.026 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:10:49.026 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:10:49.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:10:49.028 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:10:49.028 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:10:49.029 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:10:49.032 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:10:49.032 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T04:12:32.033 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.762925355328298) > weight (0.25) 2021-03-09T04:12:32.035 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4780191398047412) > weight (0.25) 2021-03-09T04:12:32.036 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5369779773828475) > weight (0.25) 2021-03-09T04:12:32.036 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T04:12:32.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T04:12:32.037 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T04:12:32.038 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T04:12:32.036+0000 7f1a80e19700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 37226) UID: 1000 2021-03-09T04:12:32.238 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:12:32.714 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6325517911633551) > weight (0.25) 2021-03-09T04:12:32.714 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T04:12:32.715 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T04:12:32.715 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T04:12:32.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T04:12:32.712+0000 7f3f7c39b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 35312) UID: 1000 2021-03-09T04:12:32.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:12:39.038 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T04:12:39.040 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:12:39.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:12:39.040 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T04:12:39.041 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T04:12:39.041 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T04:12:39.043 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T04:12:39.044 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T04:12:40.716 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T04:12:40.724 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:12:40.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:12:40.725 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T04:12:40.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T04:12:40.726 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T04:12:40.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T04:12:40.729 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-03-09T04:13:46.045 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T04:13:46.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T04:13:46.048 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T04:13:46.049 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:13:46.043+0000 7f4bc63d1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 37795) UID: 1000 2021-03-09T04:13:46.249 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:13:51.050 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T04:13:51.050 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:13:51.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:13:51.051 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T04:13:51.051 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T04:13:51.052 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T04:13:51.055 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T04:13:51.055 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T04:13:59.731 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.580089842455455) > weight (0.25) 2021-03-09T04:13:59.731 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.40902784368195744) > weight (0.25) 2021-03-09T04:13:59.731 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T04:13:59.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T04:13:59.732 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T04:13:59.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T04:13:59.731+0000 7f30b7011700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 37695) UID: 1000 2021-03-09T04:13:59.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:14:02.734 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T04:14:02.734 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:14:02.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:14:02.735 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T04:14:02.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T04:14:02.735 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T04:14:02.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T04:14:02.739 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T04:14:49.254 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:14:49.249+0000 7f901a51c700 -1 reset not still connected to 0x5588f8cb3e90 2021-03-09T04:14:49.254 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:14:49.249+0000 7f901a51c700 -1 reset not still connected to 0x5588fa39b9b0 2021-03-09T04:15:07.056 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4536434229842433) > weight (0.25) 2021-03-09T04:15:07.057 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:15:07.057 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:15:07.058 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T04:15:07.059 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:15:07.057+0000 7fb1b0c24700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 38463) UID: 1000 2021-03-09T04:15:07.259 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:15:09.058 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:15:09.059 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:15:09.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:15:09.060 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:15:09.060 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:15:09.060 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:15:09.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:15:09.063 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T04:15:55.741 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9241445803581448) > weight (0.25) 2021-03-09T04:15:55.741 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5061191682296321) > weight (0.25) 2021-03-09T04:15:55.742 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7278157430518368) > weight (0.25) 2021-03-09T04:15:55.742 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T04:15:55.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T04:15:55.743 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T04:15:55.767 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T04:15:55.761+0000 7fe50a3e0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 38272) UID: 1000 2021-03-09T04:15:55.839 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:15:55.839+0000 7f901a51c700 -1 reset not still connected to 0x5588f8cb3b50 2021-03-09T04:15:55.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:16:01.744 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T04:16:01.744 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:16:01.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:16:01.745 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T04:16:01.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T04:16:01.746 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T04:16:01.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T04:16:01.749 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T04:16:23.064 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T04:16:23.065 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T04:16:23.066 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T04:16:23.076 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:16:23.072+0000 7f1f414d8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 38746) UID: 1000 2021-03-09T04:16:23.276 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:16:30.067 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T04:16:30.067 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:16:30.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:16:30.068 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T04:16:30.068 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T04:16:30.069 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T04:16:30.072 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T04:16:30.072 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T04:16:56.281 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:16:56.277+0000 7f901a51c700 -1 reset not still connected to 0x5588fa3995f0 2021-03-09T04:16:56.281 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:16:56.277+0000 7f901a51c700 -1 reset not still connected to 0x558904ab0340 2021-03-09T04:16:56.281 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:16:56.277+0000 7f901a51c700 -1 reset not still connected to 0x558904ab1930 2021-03-09T04:16:56.282 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:16:56.277+0000 7f901a51c700 -1 reset not still connected to 0x558904ab3a80 2021-03-09T04:17:33.750 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.36708074627716814) > weight (0.25) 2021-03-09T04:17:33.757 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8057220946620794) > weight (0.25) 2021-03-09T04:17:33.757 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4682179278256384) > weight (0.25) 2021-03-09T04:17:33.758 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.30839143560314786) > weight (0.25) 2021-03-09T04:17:33.758 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T04:17:53.073 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9382765435440763) > weight (0.25) 2021-03-09T04:17:53.074 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7603094234353268) > weight (0.25) 2021-03-09T04:17:53.075 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T04:17:53.075 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T04:17:53.077 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T04:17:53.078 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T04:17:53.071+0000 7f0849b4c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 36742) UID: 1000 2021-03-09T04:17:53.277 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:18:02.080 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T04:18:02.086 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:18:02.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:18:02.087 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T04:18:02.087 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T04:18:02.089 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T04:18:02.091 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T04:18:02.092 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T04:19:11.092 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6845564884913528) > weight (0.25) 2021-03-09T04:19:11.095 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:19:11.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:19:11.097 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T04:19:11.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:19:11.093+0000 7f050afd3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 38941) UID: 1000 2021-03-09T04:19:11.298 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:19:13.098 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:19:13.098 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:19:13.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:19:13.099 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:19:13.099 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:19:13.100 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:19:13.103 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:19:13.103 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T04:19:29.759 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T04:19:29.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T04:19:29.761 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T04:19:29.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:19:29.757+0000 7fa376848700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 38079) UID: 1000 2021-03-09T04:19:29.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:19:37.761 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T04:19:37.764 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:19:37.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:19:37.766 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T04:19:37.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T04:19:37.767 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T04:19:37.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T04:19:37.772 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T04:20:20.105 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8889310225992983) > weight (0.25) 2021-03-09T04:20:20.107 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:20:20.107 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:20:20.108 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T04:20:20.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:20:20.104+0000 7fe4c727f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 39319) UID: 1000 2021-03-09T04:20:20.309 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:20:22.108 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:20:22.109 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:20:22.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:20:22.110 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:20:22.110 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:20:22.110 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:20:22.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:20:22.113 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T04:20:46.773 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5032877536279301) > weight (0.25) 2021-03-09T04:20:46.777 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.49688667171955403) > weight (0.25) 2021-03-09T04:20:46.777 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7808132218386804) > weight (0.25) 2021-03-09T04:20:46.777 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5446664162750988) > weight (0.25) 2021-03-09T04:20:46.777 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T04:22:09.778 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T04:22:09.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T04:22:09.781 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T04:22:09.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:22:09.778+0000 7fcdb3e16700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 39410) UID: 1000 2021-03-09T04:22:09.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:22:11.783 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T04:22:11.783 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:22:11.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:22:11.784 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T04:22:11.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T04:22:11.784 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T04:22:11.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T04:22:11.787 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T04:22:19.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.37795832882734215) > weight (0.25) 2021-03-09T04:22:19.115 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4996421832250705) > weight (0.25) 2021-03-09T04:22:19.115 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.35082237250114656) > weight (0.25) 2021-03-09T04:22:19.115 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T04:22:19.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T04:22:19.116 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T04:22:19.117 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T04:22:19.110+0000 7efd54be2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 38554) UID: 1000 2021-03-09T04:22:19.319 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:22:20.243 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:22:20.238+0000 7fbcec3b2700 -1 rbd::mirror::ImageReplayer: 0x55d6de867980 [3/18a549b3-61f2-46e5-af4d-1a9b02c6e0ad] start: already running 2021-03-09T04:22:21.117 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T04:22:21.118 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:22:21.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:22:21.119 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T04:22:21.119 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T04:22:21.120 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T04:22:21.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T04:22:21.128 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-03-09T04:23:42.129 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T04:23:42.133 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T04:23:42.133 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T04:23:42.134 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:23:42.129+0000 7f1649ec7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 39131) UID: 1000 2021-03-09T04:23:42.334 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:23:48.134 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T04:23:48.134 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:23:48.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:23:48.135 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T04:23:48.135 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T04:23:48.136 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T04:23:48.151 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T04:23:48.151 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T04:23:53.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8657996982501265) > weight (0.25) 2021-03-09T04:23:53.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7340777495306814) > weight (0.25) 2021-03-09T04:23:53.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.34557710957274446) > weight (0.25) 2021-03-09T04:23:53.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5858143206982392) > weight (0.25) 2021-03-09T04:23:53.790 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T04:24:58.791 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6536383139985765) > weight (0.25) 2021-03-09T04:24:58.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8848030107541182) > weight (0.25) 2021-03-09T04:24:58.794 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T04:24:58.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T04:24:58.795 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T04:24:58.796 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T04:24:58.791+0000 7f63186de700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 38837) UID: 1000 2021-03-09T04:24:58.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:24:59.797 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T04:25:00.053 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:25:00.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:25:00.054 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T04:25:00.055 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T04:25:00.055 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T04:25:00.057 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T04:25:00.058 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T04:25:31.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6253715267655608) > weight (0.25) 2021-03-09T04:25:31.156 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5457050906597232) > weight (0.25) 2021-03-09T04:25:31.156 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6054944335458899) > weight (0.25) 2021-03-09T04:25:31.156 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.47781813830759623) > weight (0.25) 2021-03-09T04:25:31.157 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T04:26:14.059 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5605736703116813) > weight (0.25) 2021-03-09T04:26:14.062 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7534262825945115) > weight (0.25) 2021-03-09T04:26:14.063 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8830529374578402) > weight (0.25) 2021-03-09T04:26:14.064 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9203046311031108) > weight (0.25) 2021-03-09T04:26:14.065 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T04:26:34.157 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.672059354357324) > weight (0.25) 2021-03-09T04:26:34.158 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9039423869644823) > weight (0.25) 2021-03-09T04:26:34.158 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.36937528899386507) > weight (0.25) 2021-03-09T04:26:34.159 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3865481352495811) > weight (0.25) 2021-03-09T04:26:34.159 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T04:27:20.066 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T04:27:20.070 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T04:27:20.070 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T04:27:20.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:27:20.069+0000 7f076ef5a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 39605) UID: 1000 2021-03-09T04:27:20.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:27:25.071 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T04:27:25.072 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:27:25.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:27:25.073 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T04:27:25.073 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T04:27:25.074 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T04:27:25.076 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T04:27:25.077 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T04:27:47.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35584591955902867) > weight (0.25) 2021-03-09T04:27:47.161 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5862650643250376) > weight (0.25) 2021-03-09T04:27:47.161 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.31918397861022074) > weight (0.25) 2021-03-09T04:27:47.161 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T04:27:47.162 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T04:27:47.162 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T04:27:47.163 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T04:27:47.160+0000 7f7d63d70700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 39704) UID: 1000 2021-03-09T04:27:47.363 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:27:54.163 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T04:27:54.164 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:27:54.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:27:54.165 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T04:27:54.165 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T04:27:54.165 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T04:27:54.168 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T04:27:54.169 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T04:28:58.078 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8766196626711316) > weight (0.25) 2021-03-09T04:28:58.080 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7764618380886983) > weight (0.25) 2021-03-09T04:28:58.080 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7550818537383281) > weight (0.25) 2021-03-09T04:28:58.080 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T04:28:58.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T04:28:58.081 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T04:28:58.082 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T04:28:58.079+0000 7f5091a0d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 39032) UID: 1000 2021-03-09T04:28:58.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:28:59.466 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:28:59.461+0000 7f901a51c700 -1 reset not still connected to 0x558903e2dc20 2021-03-09T04:28:59.472 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:28:59.465+0000 7f901a51c700 -1 reset not still connected to 0x558904ab27d0 2021-03-09T04:28:59.472 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:28:59.465+0000 7f901a51c700 -1 reset not still connected to 0x5588e7f458e0 2021-03-09T04:28:59.473 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:28:59.465+0000 7f901a51c700 -1 reset not still connected to 0x5588f923f4d0 2021-03-09T04:28:59.473 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:28:59.465+0000 7f901a51c700 -1 reset not still connected to 0x5588f923f5a0 2021-03-09T04:28:59.473 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:28:59.465+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf1d40 2021-03-09T04:28:59.473 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:28:59.465+0000 7f901a51c700 -1 reset not still connected to 0x558903e2c490 2021-03-09T04:28:59.473 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:28:59.465+0000 7f901a51c700 -1 reset not still connected to 0x558903e2dc20 2021-03-09T04:28:59.474 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:28:59.465+0000 7f901a51c700 -1 reset not still connected to 0x558904ab27d0 2021-03-09T04:29:03.082 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T04:29:03.083 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:29:03.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:29:03.084 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T04:29:03.084 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T04:29:03.084 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T04:29:03.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T04:29:03.088 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T04:29:39.170 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5553753465468396) > weight (0.25) 2021-03-09T04:29:39.172 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:29:39.172 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:29:39.173 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T04:29:39.174 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:29:39.170+0000 7fbcecbb3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 39510) UID: 1000 2021-03-09T04:29:39.272 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T04:29:39.266+0000 7f082a074700 -1 reset not still connected to 0x55a9964990c0 2021-03-09T04:29:39.272 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T04:29:39.266+0000 7f082a074700 -1 reset not still connected to 0x55aa12b1ec30 2021-03-09T04:29:39.374 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:29:43.174 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:29:43.175 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:29:43.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:29:43.176 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:29:43.176 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:29:43.178 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:29:43.180 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:29:43.181 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T04:30:08.089 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7032442927816321) > weight (0.25) 2021-03-09T04:30:08.096 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.745765831641225) > weight (0.25) 2021-03-09T04:30:08.096 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2614509131081033) > weight (0.25) 2021-03-09T04:30:08.097 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.40916288348408314) > weight (0.25) 2021-03-09T04:30:08.097 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T04:31:36.181 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6581826160809363) > weight (0.25) 2021-03-09T04:31:36.182 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4285600712505836) > weight (0.25) 2021-03-09T04:31:36.183 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T04:31:36.183 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T04:31:36.184 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T04:31:36.185 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T04:31:36.180+0000 7fb6f2207700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 39226) UID: 1000 2021-03-09T04:31:36.385 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:31:38.185 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T04:31:38.186 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:31:38.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:31:38.186 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T04:31:38.187 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T04:31:38.187 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T04:31:38.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T04:31:38.190 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T04:31:59.098 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T04:31:59.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T04:31:59.129 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T04:31:59.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:31:59.128+0000 7f30d2b48700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 39991) UID: 1000 2021-03-09T04:31:59.330 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:32:05.131 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T04:32:05.133 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:32:05.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:32:05.134 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T04:32:05.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T04:32:05.135 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T04:32:05.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T04:32:05.140 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T04:33:27.191 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5412034184373793) > weight (0.25) 2021-03-09T04:33:27.193 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7752570064922946) > weight (0.25) 2021-03-09T04:33:27.194 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T04:33:27.194 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T04:33:27.195 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T04:33:27.196 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T04:33:27.190+0000 7f50565f6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 40375) UID: 1000 2021-03-09T04:33:27.396 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:33:30.196 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T04:33:30.196 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:33:30.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:33:30.197 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T04:33:30.197 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T04:33:30.198 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T04:33:30.201 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T04:33:30.201 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T04:34:04.141 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T04:34:04.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T04:34:04.154 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T04:34:04.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:34:04.150+0000 7fefe76ef700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 40466) UID: 1000 2021-03-09T04:34:04.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:34:08.155 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T04:34:08.157 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:34:08.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:34:08.158 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T04:34:08.159 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T04:34:08.160 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T04:34:08.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T04:34:08.164 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T04:35:18.202 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4067985748811095) > weight (0.25) 2021-03-09T04:35:18.235 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3454721416444242) > weight (0.25) 2021-03-09T04:35:18.236 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7882881555388292) > weight (0.25) 2021-03-09T04:35:18.236 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T04:35:18.237 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T04:35:18.237 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T04:35:18.238 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T04:35:18.233+0000 7f1e3ac8b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 40090) UID: 1000 2021-03-09T04:35:18.439 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:35:25.238 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T04:35:25.240 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:35:25.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:35:25.241 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T04:35:25.241 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T04:35:25.241 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T04:35:25.244 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T04:35:25.245 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T04:35:49.165 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2929291986440892) > weight (0.25) 2021-03-09T04:35:49.167 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5847893120584027) > weight (0.25) 2021-03-09T04:35:49.167 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5988458684061356) > weight (0.25) 2021-03-09T04:35:49.167 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9156077246480994) > weight (0.25) 2021-03-09T04:35:49.167 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T04:37:13.246 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6182012805454421) > weight (0.25) 2021-03-09T04:37:13.250 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:37:13.261 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:37:13.262 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T04:37:13.263 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:37:13.259+0000 7fa1023e3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 40281) UID: 1000 2021-03-09T04:37:13.462 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:37:17.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.39758039679140034) > weight (0.25) 2021-03-09T04:37:17.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6467912951061219) > weight (0.25) 2021-03-09T04:37:17.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4235758172262699) > weight (0.25) 2021-03-09T04:37:17.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.47772947427895196) > weight (0.25) 2021-03-09T04:37:17.169 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T04:37:20.262 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:37:20.267 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:37:20.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:37:20.268 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:37:20.268 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:37:20.269 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:37:20.271 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:37:20.272 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T04:38:20.273 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9970890904188608) > weight (0.25) 2021-03-09T04:38:20.273 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:38:20.274 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:38:20.275 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T04:38:20.276 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:38:20.273+0000 7f9b6605a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 40853) UID: 1000 2021-03-09T04:38:20.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:38:26.170 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8702162217312547) > weight (0.25) 2021-03-09T04:38:26.192 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T04:38:26.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T04:38:26.193 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T04:38:26.194 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T04:38:26.194 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:38:26.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T04:38:26.189+0000 7ff2e0ee1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 38645) UID: 1000 2021-03-09T04:38:26.394 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:38:28.276 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:38:28.278 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:38:28.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:38:28.279 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:38:28.279 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:38:28.287 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:38:28.290 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:38:28.290 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T04:38:32.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:38:32.299 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T04:38:32.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T04:38:32.302 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T04:38:32.306 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T04:38:32.306 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T04:40:13.291 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29350205830776577) > weight (0.25) 2021-03-09T04:40:13.293 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8466129147122168) > weight (0.25) 2021-03-09T04:40:13.294 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5561787400853831) > weight (0.25) 2021-03-09T04:40:13.294 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6175999561716734) > weight (0.25) 2021-03-09T04:40:13.294 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T04:40:17.307 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.28440156815824225) > weight (0.25) 2021-03-09T04:40:17.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6508441709772411) > weight (0.25) 2021-03-09T04:40:17.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9237397165670067) > weight (0.25) 2021-03-09T04:40:17.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.28524471263227447) > weight (0.25) 2021-03-09T04:40:17.308 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T04:41:30.310 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3254034805829562) > weight (0.25) 2021-03-09T04:41:30.311 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9075364582816758) > weight (0.25) 2021-03-09T04:41:30.311 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8840428899247434) > weight (0.25) 2021-03-09T04:41:30.311 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5869393766909936) > weight (0.25) 2021-03-09T04:41:30.312 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T04:41:42.296 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6890299849139935) > weight (0.25) 2021-03-09T04:41:42.296 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:41:42.297 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:41:42.298 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T04:41:42.299 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:41:42.294+0000 7fbde11a7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 40944) UID: 1000 2021-03-09T04:41:42.499 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:41:46.299 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:41:46.299 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:41:46.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:41:46.300 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:41:46.301 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:41:46.301 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:41:46.304 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:41:46.305 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T04:43:06.306 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T04:43:06.337 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T04:43:06.337 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T04:43:06.338 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:43:06.332+0000 7faf7a680700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 39796) UID: 1000 2021-03-09T04:43:06.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:43:13.339 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T04:43:13.340 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:43:13.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:43:13.341 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T04:43:13.341 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T04:43:13.343 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T04:43:13.346 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T04:43:13.346 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T04:43:22.313 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.38647605713319333) > weight (0.25) 2021-03-09T04:43:22.313 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T04:43:22.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T04:43:22.314 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T04:43:22.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T04:43:22.312+0000 7f9489b49700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 41035) UID: 1000 2021-03-09T04:43:22.515 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:43:28.315 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T04:43:28.316 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:43:28.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:43:28.317 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T04:43:28.317 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T04:43:28.317 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T04:43:28.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T04:43:28.320 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-03-09T04:44:41.347 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T04:44:41.349 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T04:44:41.350 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T04:44:41.351 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:44:41.347+0000 7f0d91611700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 41229) UID: 1000 2021-03-09T04:44:41.550 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:44:47.322 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6434085870932023) > weight (0.25) 2021-03-09T04:44:47.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7300612672091511) > weight (0.25) 2021-03-09T04:44:47.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7068192822224294) > weight (0.25) 2021-03-09T04:44:47.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2787128996397903) > weight (0.25) 2021-03-09T04:44:47.346 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T04:44:48.350 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T04:44:48.351 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:44:48.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:44:48.352 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T04:44:48.352 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T04:44:48.354 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T04:44:48.357 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T04:44:48.357 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T04:46:28.346 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32731827978796735) > weight (0.25) 2021-03-09T04:46:28.350 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7682808014208227) > weight (0.25) 2021-03-09T04:46:28.350 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6204661570250344) > weight (0.25) 2021-03-09T04:46:28.350 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T04:46:28.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T04:46:28.351 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T04:46:28.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T04:46:28.345+0000 7fa523de1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 40181) UID: 1000 2021-03-09T04:46:28.553 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:46:29.358 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4011758402470663) > weight (0.25) 2021-03-09T04:46:29.359 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.30610285813556337) > weight (0.25) 2021-03-09T04:46:29.359 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7216079273170282) > weight (0.25) 2021-03-09T04:46:29.359 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8941065984723026) > weight (0.25) 2021-03-09T04:46:29.359 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-03-09T04:46:31.352 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T04:46:31.353 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:46:31.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:46:31.353 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T04:46:31.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T04:46:31.354 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T04:46:31.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T04:46:31.357 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T04:48:05.361 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T04:48:05.422 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T04:48:05.423 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T04:48:05.424 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:48:05.419+0000 7f5d3936c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 41419) UID: 1000 2021-03-09T04:48:05.623 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:48:06.424 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T04:48:06.425 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:48:06.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:48:06.425 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T04:48:06.426 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T04:48:06.426 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T04:48:06.429 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T04:48:06.429 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T04:48:20.358 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6503211627605574) > weight (0.25) 2021-03-09T04:48:20.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4929600514164916) > weight (0.25) 2021-03-09T04:48:20.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5185324052593748) > weight (0.25) 2021-03-09T04:48:20.360 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T04:48:20.361 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T04:48:20.362 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T04:48:20.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T04:48:20.359+0000 7f313c05a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 41515) UID: 1000 2021-03-09T04:48:20.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:48:24.363 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T04:48:24.370 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:48:24.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:48:24.371 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T04:48:24.371 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T04:48:24.372 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T04:48:24.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T04:48:24.376 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T04:49:22.430 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T04:49:22.434 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T04:49:22.434 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T04:49:22.435 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:49:22.430+0000 7f8596b36700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 41615) UID: 1000 2021-03-09T04:49:22.636 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:49:24.436 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T04:49:24.436 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:49:24.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:49:24.437 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T04:49:24.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T04:49:24.438 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T04:49:24.441 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T04:49:24.441 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T04:49:36.377 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9595117962709329) > weight (0.25) 2021-03-09T04:49:36.380 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8644073249364895) > weight (0.25) 2021-03-09T04:49:36.381 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.732728766720781) > weight (0.25) 2021-03-09T04:49:36.381 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8531439308688359) > weight (0.25) 2021-03-09T04:49:36.381 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-03-09T04:50:33.442 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7515200165857759) > weight (0.25) 2021-03-09T04:50:33.445 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.305140164564512) > weight (0.25) 2021-03-09T04:50:33.446 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T04:50:33.446 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T04:50:33.447 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T04:50:33.448 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T04:50:33.445+0000 7f7155595700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 40565) UID: 1000 2021-03-09T04:50:33.648 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:50:41.447 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T04:50:41.451 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:50:41.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:50:41.452 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T04:50:41.452 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T04:50:41.453 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T04:50:41.456 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T04:50:41.456 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T04:50:57.383 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T04:50:57.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T04:50:57.387 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T04:50:57.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:50:57.384+0000 7f62cc61c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 40656) UID: 1000 2021-03-09T04:50:57.588 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:50:58.388 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T04:50:58.388 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:50:58.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:50:58.389 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T04:50:58.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T04:50:58.390 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T04:50:58.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T04:50:58.393 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T04:52:18.394 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7772772962980926) > weight (0.25) 2021-03-09T04:52:18.395 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4984956563753212) > weight (0.25) 2021-03-09T04:52:18.396 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T04:52:18.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T04:52:18.397 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T04:52:18.397 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T04:52:18.397 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:52:18.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T04:52:18.391+0000 7efee96a0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 39888) UID: 1000 2021-03-09T04:52:18.546 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:52:18.544+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf3e90 2021-03-09T04:52:18.598 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:52:19.755 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T04:52:19.751+0000 7f082a074700 -1 reset not still connected to 0x55a9a13aab60 2021-03-09T04:52:19.756 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T04:52:19.751+0000 7f082a074700 -1 reset not still connected to 0x55a9f121ab60 2021-03-09T04:52:24.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:52:24.500 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T04:52:24.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T04:52:24.501 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T04:52:24.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T04:52:24.504 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T04:52:40.457 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6183462825654514) > weight (0.25) 2021-03-09T04:52:40.461 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3809389761771166) > weight (0.25) 2021-03-09T04:52:40.462 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.94218318195936) > weight (0.25) 2021-03-09T04:52:40.462 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T04:52:40.463 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T04:52:40.464 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T04:52:40.465 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T04:52:40.459+0000 7fed8073f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 40759) UID: 1000 2021-03-09T04:52:40.665 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:52:44.465 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T04:52:44.465 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:52:44.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:52:44.467 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T04:52:44.467 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T04:52:44.468 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T04:52:44.471 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T04:52:44.472 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T04:53:46.505 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4774250327109377) > weight (0.25) 2021-03-09T04:53:46.509 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5455725722675282) > weight (0.25) 2021-03-09T04:53:46.509 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6902511415706958) > weight (0.25) 2021-03-09T04:53:46.509 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T04:53:46.510 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T04:53:46.510 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T04:53:46.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T04:53:46.506+0000 7f466f379700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 41706) UID: 1000 2021-03-09T04:53:46.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:53:47.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:53:47.658+0000 7f874c4ed700 -1 rbd::mirror::ImageReplayer: 0x55e75ffe6000 [3/59893ab7-fc80-4b4a-85c5-aa491edb65bc] start: already running 2021-03-09T04:53:47.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:53:47.658+0000 7f874c4ed700 -1 rbd::mirror::ImageReplayer: 0x55e75be47900 [3/90629254-6fb9-41d9-b690-e94fb222ffa6] start: already running 2021-03-09T04:53:47.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:53:47.658+0000 7f874c4ed700 -1 rbd::mirror::ImageReplayer: 0x55e75be47400 [3/a27c2afb-1b52-4788-afd8-b540c509149b] start: already running 2021-03-09T04:53:47.670 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T04:53:47.662+0000 7f082a074700 -1 reset not still connected to 0x55a996496a90 2021-03-09T04:53:47.673 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T04:53:47.666+0000 7f082a074700 -1 reset not still connected to 0x55a9975a3e90 2021-03-09T04:53:47.919 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T04:53:47.913+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad0226d80 2021-03-09T04:53:47.930 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:53:47.929+0000 7f901a51c700 -1 reset not still connected to 0x5588e7f428f0 2021-03-09T04:53:47.930 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:53:47.929+0000 7f901a51c700 -1 reset not still connected to 0x5588e7f43520 2021-03-09T04:53:47.930 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:53:47.929+0000 7f901a51c700 -1 reset not still connected to 0x5588e7f44970 2021-03-09T04:53:47.931 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:53:47.929+0000 7f901a51c700 -1 reset not still connected to 0x5588f923cdd0 2021-03-09T04:53:47.932 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:53:47.929+0000 7f901a51c700 -1 reset not still connected to 0x5588f923d450 2021-03-09T04:53:47.932 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:53:47.929+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf2970 2021-03-09T04:53:49.511 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T04:53:49.512 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:53:49.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:53:49.513 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T04:53:49.514 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T04:53:49.515 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T04:53:49.518 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T04:53:49.519 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T04:54:09.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4561986398665845) > weight (0.25) 2021-03-09T04:54:09.475 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T04:54:09.476 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T04:54:09.477 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T04:54:09.478 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T04:54:09.473+0000 7f0c9a5a9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 41137) UID: 1000 2021-03-09T04:54:09.677 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:54:10.619 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:54:10.613+0000 7f3280b2d700 -1 rbd::mirror::ImageReplayer: 0x55c2ccc98300 [3/18a549b3-61f2-46e5-af4d-1a9b02c6e0ad] start: already running 2021-03-09T04:54:16.478 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T04:54:16.479 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:54:16.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:54:16.479 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T04:54:16.480 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T04:54:16.480 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T04:54:16.483 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T04:54:16.483 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T04:55:22.520 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.26211121551023475) > weight (0.25) 2021-03-09T04:55:22.522 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2682270800017351) > weight (0.25) 2021-03-09T04:55:22.523 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.34166824294634746) > weight (0.25) 2021-03-09T04:55:22.523 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4152227710002441) > weight (0.25) 2021-03-09T04:55:22.523 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T04:55:25.484 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T04:55:25.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T04:55:25.486 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T04:55:25.486 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T04:55:25.486 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:55:25.487 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T04:55:25.484+0000 7f3281b2f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 41805) UID: 1000 2021-03-09T04:55:25.687 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:55:31.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:55:31.589 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T04:55:31.589 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T04:55:31.589 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T04:55:31.593 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T04:55:31.593 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T04:55:56.784 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T04:55:56.782+0000 7f901a51c700 -1 reset not still connected to 0x5588f923f400 2021-03-09T04:56:55.524 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T04:56:55.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T04:56:55.527 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T04:56:55.528 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:56:55.522+0000 7f874d4ef700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 41988) UID: 1000 2021-03-09T04:56:55.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:56:59.594 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.557549994950406) > weight (0.25) 2021-03-09T04:56:59.595 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.47475683782639555) > weight (0.25) 2021-03-09T04:56:59.595 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T04:56:59.595 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T04:56:59.596 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T04:56:59.597 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T04:56:59.590+0000 7f951213e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 41897) UID: 1000 2021-03-09T04:56:59.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:57:00.528 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T04:57:00.528 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:57:00.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:57:00.529 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T04:57:00.530 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T04:57:00.530 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T04:57:00.533 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T04:57:00.533 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T04:57:03.597 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T04:57:03.597 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:57:03.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:57:03.598 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T04:57:03.599 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T04:57:03.599 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T04:57:03.602 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T04:57:03.602 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T04:58:22.534 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7945452990967945) > weight (0.25) 2021-03-09T04:58:22.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3152013649583577) > weight (0.25) 2021-03-09T04:58:22.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9348641890982746) > weight (0.25) 2021-03-09T04:58:22.537 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7087111862516452) > weight (0.25) 2021-03-09T04:58:22.537 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T04:58:55.604 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6507780225750677) > weight (0.25) 2021-03-09T04:58:55.605 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8146739075765292) > weight (0.25) 2021-03-09T04:58:55.605 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.641049846128765) > weight (0.25) 2021-03-09T04:58:55.605 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7607781984018368) > weight (0.25) 2021-03-09T04:58:55.605 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T04:59:29.538 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T04:59:29.541 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T04:59:29.542 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T04:59:29.543 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T04:59:29.536+0000 7f3e510cf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 42567) UID: 1000 2021-03-09T04:59:29.742 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T04:59:31.543 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T04:59:31.544 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T04:59:31.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T04:59:31.545 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T04:59:31.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T04:59:31.546 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T04:59:31.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T04:59:31.571 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T05:00:18.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4978544453935232) > weight (0.25) 2021-03-09T05:00:18.608 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T05:00:18.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T05:00:18.610 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T05:00:18.611 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T05:00:18.607+0000 7f6dc92f7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 42380) UID: 1000 2021-03-09T05:00:18.811 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:00:26.611 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T05:00:26.612 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:00:26.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:00:26.613 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T05:00:26.614 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T05:00:26.614 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T05:00:26.618 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T05:00:26.618 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-03-09T05:01:10.572 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5066122755843491) > weight (0.25) 2021-03-09T05:01:10.576 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:01:10.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:01:10.579 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T05:01:10.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:01:10.574+0000 7fe8717d5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 41320) UID: 1000 2021-03-09T05:01:10.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:01:12.579 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:01:12.585 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:01:12.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:01:12.587 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:01:12.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:01:12.588 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:01:12.591 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:01:12.592 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T05:02:24.620 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2739649266563391) > weight (0.25) 2021-03-09T05:02:24.624 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5144962304725866) > weight (0.25) 2021-03-09T05:02:24.625 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4301665255841848) > weight (0.25) 2021-03-09T05:02:24.625 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6583417734685637) > weight (0.25) 2021-03-09T05:02:24.626 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T05:03:09.593 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8525571620436158) > weight (0.25) 2021-03-09T05:03:09.594 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9868805766929285) > weight (0.25) 2021-03-09T05:03:09.594 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8372570258437206) > weight (0.25) 2021-03-09T05:03:09.594 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T05:03:09.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T05:03:09.596 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T05:03:09.597 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T05:03:09.592+0000 7f351383a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 42281) UID: 1000 2021-03-09T05:03:09.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:03:14.597 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T05:03:14.597 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:03:14.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:03:14.598 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T05:03:14.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T05:03:14.599 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T05:03:14.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T05:03:14.602 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T05:03:51.627 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T05:03:51.628 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T05:03:51.628 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T05:03:51.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T05:03:51.623+0000 7f40da81f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 42474) UID: 1000 2021-03-09T05:03:51.830 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:04:00.629 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T05:04:00.630 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:04:00.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:04:00.631 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T05:04:00.631 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T05:04:00.632 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T05:04:00.635 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T05:04:00.635 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T05:04:26.604 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9828967123726361) > weight (0.25) 2021-03-09T05:04:26.604 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.429543567264005) > weight (0.25) 2021-03-09T05:04:26.605 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.26346763910769566) > weight (0.25) 2021-03-09T05:04:26.605 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8217914814324534) > weight (0.25) 2021-03-09T05:04:26.605 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-03-09T05:05:01.636 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29863112247499923) > weight (0.25) 2021-03-09T05:05:01.637 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5225379722956599) > weight (0.25) 2021-03-09T05:05:01.637 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7084750827081853) > weight (0.25) 2021-03-09T05:05:01.637 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9139667744487537) > weight (0.25) 2021-03-09T05:05:01.638 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-03-09T05:05:41.606 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2702803440524604) > weight (0.25) 2021-03-09T05:05:41.607 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8433356058374933) > weight (0.25) 2021-03-09T05:05:41.607 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T05:05:41.608 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T05:05:41.608 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T05:05:41.609 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T05:05:41.605+0000 7f8e9eb03700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 42089) UID: 1000 2021-03-09T05:05:41.691 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:05:41.681+0000 7f082a074700 -1 reset not still connected to 0x55a9f121d190 2021-03-09T05:05:41.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:05:45.610 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T05:05:45.610 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:05:45.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:05:45.611 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T05:05:45.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T05:05:45.612 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T05:05:45.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T05:05:45.615 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T05:06:27.638 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T05:06:27.640 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T05:06:27.640 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T05:06:27.641 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T05:06:27.632+0000 7fbc9df4e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 43150) UID: 1000 2021-03-09T05:06:27.842 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:06:28.641 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T05:06:28.641 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:06:28.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:06:28.642 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T05:06:28.642 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T05:06:28.643 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T05:06:28.645 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T05:06:28.646 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T05:07:02.616 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5379082028158425) > weight (0.25) 2021-03-09T05:07:02.617 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:07:02.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:07:02.618 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T05:07:02.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:07:02.616+0000 7f451d00b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 42948) UID: 1000 2021-03-09T05:07:02.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:07:07.619 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:07:07.620 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:07:07.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:07:07.621 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:07:07.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:07:07.621 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:07:07.624 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:07:07.625 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-03-09T05:07:37.647 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T05:07:37.648 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T05:07:37.649 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T05:07:37.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T05:07:37.643+0000 7f4f7444d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 43343) UID: 1000 2021-03-09T05:07:37.850 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:07:39.650 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T05:07:39.650 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:07:39.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:07:39.651 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T05:07:39.651 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T05:07:39.652 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T05:07:39.654 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T05:07:39.654 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T05:08:33.625 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.27302016581540345) > weight (0.25) 2021-03-09T05:08:33.626 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:08:33.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:08:33.627 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T05:08:33.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:08:33.622+0000 7f910a64f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 43434) UID: 1000 2021-03-09T05:08:33.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:08:38.628 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:08:38.628 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:08:38.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:08:38.629 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:08:38.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:08:38.630 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:08:38.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:08:38.633 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-03-09T05:08:43.655 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3428720937342721) > weight (0.25) 2021-03-09T05:08:43.656 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9444100272882119) > weight (0.25) 2021-03-09T05:08:43.656 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7186185911924815) > weight (0.25) 2021-03-09T05:08:43.656 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.34818453878984523) > weight (0.25) 2021-03-09T05:08:43.656 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T05:09:05.060 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:09:05.055+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf2080 2021-03-09T05:09:05.061 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:09:05.055+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf2560 2021-03-09T05:09:05.061 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:09:05.055+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf30c0 2021-03-09T05:09:05.061 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:09:05.055+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf3a80 2021-03-09T05:09:05.061 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:09:05.055+0000 7f901a51c700 -1 reset not still connected to 0x558904ab2970 2021-03-09T05:10:16.635 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7068041928591694) > weight (0.25) 2021-03-09T05:10:16.636 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:10:16.637 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:10:16.638 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T05:10:16.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:10:16.637+0000 7f1166e7f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 43628) UID: 1000 2021-03-09T05:10:16.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:10:18.639 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:10:18.640 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:10:18.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:10:18.641 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:10:18.641 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:10:18.642 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:10:18.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:10:18.645 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T05:10:26.854 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:10:26.848+0000 7f082a074700 -1 reset not still connected to 0x55a991fffcf0 2021-03-09T05:10:26.854 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:10:26.848+0000 7f082a074700 -1 reset not still connected to 0x55a99af619b0 2021-03-09T05:10:34.658 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9667174338223564) > weight (0.25) 2021-03-09T05:10:34.658 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8721012488598847) > weight (0.25) 2021-03-09T05:10:34.658 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9386575551662262) > weight (0.25) 2021-03-09T05:10:34.659 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8984281599325721) > weight (0.25) 2021-03-09T05:10:34.659 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T05:10:47.241 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:10:47.228+0000 7f901a51c700 -1 reset not still connected to 0x5588f923c4e0 2021-03-09T05:10:47.241 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:10:47.228+0000 7f901a51c700 -1 reset not still connected to 0x5588f96f2cb0 2021-03-09T05:10:47.242 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:10:47.228+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf05b0 2021-03-09T05:10:47.242 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:10:47.232+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf3cf0 2021-03-09T05:10:47.242 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:10:47.232+0000 7f901a51c700 -1 reset not still connected to 0x558903e2ad00 2021-03-09T05:11:30.646 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.43274250265080527) > weight (0.25) 2021-03-09T05:11:30.647 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9083113654120073) > weight (0.25) 2021-03-09T05:11:30.647 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5492984924622785) > weight (0.25) 2021-03-09T05:11:30.647 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8173485856459306) > weight (0.25) 2021-03-09T05:11:30.647 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T05:12:18.661 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7351918148360082) > weight (0.25) 2021-03-09T05:12:18.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.37559951177502493) > weight (0.25) 2021-03-09T05:12:18.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7507774479778749) > weight (0.25) 2021-03-09T05:12:18.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8169814459361437) > weight (0.25) 2021-03-09T05:12:18.663 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T05:13:18.649 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9894152918350054) > weight (0.25) 2021-03-09T05:13:18.650 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8794637806619569) > weight (0.25) 2021-03-09T05:13:18.650 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.36338732323533884) > weight (0.25) 2021-03-09T05:13:18.651 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.38213474427048877) > weight (0.25) 2021-03-09T05:13:18.651 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T05:13:21.664 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6984891700117329) > weight (0.25) 2021-03-09T05:13:21.664 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T05:13:21.665 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T05:13:21.666 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T05:13:21.667 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T05:13:21.661+0000 7f0cd197a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 42857) UID: 1000 2021-03-09T05:13:21.867 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:13:28.667 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T05:13:28.668 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:13:28.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:13:28.669 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T05:13:28.669 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T05:13:28.669 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T05:13:28.672 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T05:13:28.673 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T05:14:48.653 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8592429270980612) > weight (0.25) 2021-03-09T05:14:48.653 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:14:48.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:14:48.655 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T05:14:48.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:14:48.652+0000 7fcda545f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 43728) UID: 1000 2021-03-09T05:14:48.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:14:52.656 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:14:52.656 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:14:52.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:14:52.657 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:14:52.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:14:52.658 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:14:52.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:14:52.661 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T05:14:58.673 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.28637636955175116) > weight (0.25) 2021-03-09T05:14:58.674 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7866868478647098) > weight (0.25) 2021-03-09T05:14:58.674 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8464580807865028) > weight (0.25) 2021-03-09T05:14:58.674 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T05:14:58.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T05:14:58.675 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T05:14:58.676 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T05:14:58.672+0000 7fa9910d2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 42188) UID: 1000 2021-03-09T05:14:58.876 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:15:07.676 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T05:15:07.676 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:15:07.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:15:07.677 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T05:15:07.677 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T05:15:07.678 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T05:15:07.681 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T05:15:07.682 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T05:15:59.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.34254622511812394) > weight (0.25) 2021-03-09T05:15:59.663 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:15:59.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:15:59.665 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T05:15:59.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:15:59.659+0000 7fba950b8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 43922) UID: 1000 2021-03-09T05:15:59.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:16:03.665 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:16:03.666 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:16:03.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:16:03.667 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:16:03.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:16:03.667 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:16:03.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:16:03.671 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-03-09T05:16:16.683 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T05:16:16.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T05:16:16.684 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T05:16:16.685 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T05:16:16.682+0000 7f88cb9c9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 43534) UID: 1000 2021-03-09T05:16:16.887 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:16:17.685 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T05:16:17.686 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:16:17.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:16:17.687 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T05:16:17.687 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T05:16:17.687 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T05:16:17.690 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T05:16:17.690 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T05:17:18.672 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T05:17:18.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T05:17:18.675 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T05:17:18.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T05:17:18.669+0000 7f7b58130700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 42757) UID: 1000 2021-03-09T05:17:18.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:17:20.675 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T05:17:20.676 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:17:20.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:17:20.677 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T05:17:20.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T05:17:20.677 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T05:17:20.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T05:17:20.680 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-03-09T05:17:22.691 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T05:17:22.692 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T05:17:22.692 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T05:17:22.693 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T05:17:22.689+0000 7f46f16bf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 44215) UID: 1000 2021-03-09T05:17:22.893 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:17:28.909 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:17:28.901+0000 7f082a074700 -1 reset not still connected to 0x55a996498490 2021-03-09T05:17:28.910 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:17:28.901+0000 7f082a074700 -1 reset not still connected to 0x55a99d053ad0 2021-03-09T05:17:28.910 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:17:28.901+0000 7f082a074700 -1 reset not still connected to 0x55a99f964150 2021-03-09T05:17:29.693 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T05:17:29.693 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:17:29.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:17:29.694 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T05:17:29.694 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T05:17:29.695 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T05:17:29.697 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T05:17:29.697 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T05:18:31.681 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T05:18:31.682 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T05:18:31.682 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T05:18:31.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T05:18:31.680+0000 7f3dc6986700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44310) UID: 1000 2021-03-09T05:18:31.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:18:32.683 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T05:18:32.683 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:18:32.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:18:32.684 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T05:18:32.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T05:18:32.685 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T05:18:32.688 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T05:18:32.688 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T05:18:51.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.498016316200267) > weight (0.25) 2021-03-09T05:18:51.699 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5508633172564863) > weight (0.25) 2021-03-09T05:18:51.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6675590263439996) > weight (0.25) 2021-03-09T05:18:51.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5122433361555437) > weight (0.25) 2021-03-09T05:18:51.700 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-03-09T05:20:12.689 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9322990106973007) > weight (0.25) 2021-03-09T05:20:12.690 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2582641881829214) > weight (0.25) 2021-03-09T05:20:12.690 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3984813217132612) > weight (0.25) 2021-03-09T05:20:12.690 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.34959300957350115) > weight (0.25) 2021-03-09T05:20:12.690 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-03-09T05:20:26.701 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7979316828886363) > weight (0.25) 2021-03-09T05:20:26.702 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6787202996548283) > weight (0.25) 2021-03-09T05:20:26.702 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9850530509017974) > weight (0.25) 2021-03-09T05:20:26.702 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5305686723223103) > weight (0.25) 2021-03-09T05:20:26.703 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-03-09T05:21:07.898 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:21:07.889+0000 7f082a074700 -1 reset not still connected to 0x55a98f9cc1a0 2021-03-09T05:21:31.692 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.505167675345678) > weight (0.25) 2021-03-09T05:21:31.692 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.38957495471518444) > weight (0.25) 2021-03-09T05:21:31.693 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8544951993631723) > weight (0.25) 2021-03-09T05:21:31.693 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8882970661928256) > weight (0.25) 2021-03-09T05:21:31.693 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-03-09T05:22:01.704 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9105763353472978) > weight (0.25) 2021-03-09T05:22:01.704 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4700283466264582) > weight (0.25) 2021-03-09T05:22:01.704 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T05:22:01.705 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T05:22:01.706 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T05:22:01.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T05:22:01.705+0000 7f38f3bf0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 42658) UID: 1000 2021-03-09T05:22:01.907 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:22:06.706 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T05:22:06.707 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:22:06.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:22:06.707 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T05:22:06.708 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T05:22:06.708 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T05:22:06.711 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T05:22:06.711 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T05:23:06.694 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T05:23:06.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T05:23:06.698 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T05:23:06.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T05:23:06.696+0000 7f632f270700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44501) UID: 1000 2021-03-09T05:23:06.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:23:13.699 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T05:23:13.700 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:23:13.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:23:13.700 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T05:23:13.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T05:23:13.701 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T05:23:13.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T05:23:13.704 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T05:23:47.713 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5034879507861463) > weight (0.25) 2021-03-09T05:23:47.761 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7193340112920664) > weight (0.25) 2021-03-09T05:23:47.761 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9580282960360174) > weight (0.25) 2021-03-09T05:23:47.762 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.49368479934693354) > weight (0.25) 2021-03-09T05:23:47.762 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T05:23:59.937 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:23:59.931+0000 7f082a074700 -1 reset not still connected to 0x55a9f121bee0 2021-03-09T05:24:51.763 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6602831571625554) > weight (0.25) 2021-03-09T05:24:51.764 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9228411932203775) > weight (0.25) 2021-03-09T05:24:51.765 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6899532138479291) > weight (0.25) 2021-03-09T05:24:51.765 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5321031773772665) > weight (0.25) 2021-03-09T05:24:51.765 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T05:25:03.705 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.692628186110505) > weight (0.25) 2021-03-09T05:25:03.706 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7755748414505513) > weight (0.25) 2021-03-09T05:25:03.706 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7868791352771151) > weight (0.25) 2021-03-09T05:25:03.707 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8528881939358105) > weight (0.25) 2021-03-09T05:25:03.707 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T05:26:14.767 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T05:26:14.831 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T05:26:14.832 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T05:26:14.832 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T05:26:14.825+0000 7fb0a32b4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 44409) UID: 1000 2021-03-09T05:26:15.033 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:26:17.833 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T05:26:17.839 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:26:17.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:26:17.839 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T05:26:17.840 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T05:26:17.841 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T05:26:17.844 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T05:26:17.844 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T05:26:55.708 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8263905187363866) > weight (0.25) 2021-03-09T05:26:55.709 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5775510797293911) > weight (0.25) 2021-03-09T05:26:55.710 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.43263428824609973) > weight (0.25) 2021-03-09T05:26:55.710 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.38199582610438887) > weight (0.25) 2021-03-09T05:26:55.710 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T05:27:44.845 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9649182659470197) > weight (0.25) 2021-03-09T05:27:44.848 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.883329508580716) > weight (0.25) 2021-03-09T05:27:44.848 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T05:27:44.849 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T05:27:44.850 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T05:27:44.851 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T05:27:44.847+0000 7fd500eb9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 44602) UID: 1000 2021-03-09T05:27:45.051 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:27:51.851 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T05:27:51.851 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:27:51.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:27:51.852 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T05:27:51.852 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T05:27:51.853 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T05:27:51.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T05:27:51.858 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T05:28:20.711 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6789314597411943) > weight (0.25) 2021-03-09T05:28:20.712 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:28:20.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:28:20.713 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T05:28:20.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:28:20.711+0000 7f90dfc42700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 44116) UID: 1000 2021-03-09T05:28:20.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:28:22.714 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:28:22.715 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:28:22.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:28:22.716 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:28:22.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:28:22.716 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:28:22.719 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:28:22.719 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T05:29:22.859 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T05:29:22.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T05:29:22.862 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T05:29:22.863 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T05:29:22.858+0000 7f91742eb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 44798) UID: 1000 2021-03-09T05:29:23.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:29:23.863 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T05:29:23.863 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:29:23.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:29:23.864 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T05:29:23.864 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T05:29:23.865 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T05:29:23.867 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T05:29:23.868 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T05:30:01.720 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T05:30:01.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T05:30:01.732 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T05:30:01.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T05:30:01.729+0000 7fc391bc2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 44693) UID: 1000 2021-03-09T05:30:01.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:30:09.734 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T05:30:09.734 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:30:09.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:30:09.735 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T05:30:09.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T05:30:09.736 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T05:30:09.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T05:30:09.739 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-03-09T05:30:50.868 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.41900707768731993) > weight (0.25) 2021-03-09T05:30:50.871 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7007074465824381) > weight (0.25) 2021-03-09T05:30:50.871 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T05:30:50.872 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T05:30:50.873 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T05:30:50.873 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T05:30:50.872+0000 7f97b2e21700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 44890) UID: 1000 2021-03-09T05:30:51.074 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:30:58.874 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T05:30:58.875 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:30:58.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:30:58.875 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T05:30:58.876 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T05:30:58.876 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T05:30:58.880 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T05:30:58.880 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T05:31:47.740 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7747910216581035) > weight (0.25) 2021-03-09T05:31:47.741 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9290159905871108) > weight (0.25) 2021-03-09T05:31:47.741 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.2839984219137238) > weight (0.25) 2021-03-09T05:31:47.741 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2969688211465715) > weight (0.25) 2021-03-09T05:31:47.741 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T05:32:22.881 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T05:32:22.882 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T05:32:22.882 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T05:32:22.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T05:32:22.879+0000 7f880e9bd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 45081) UID: 1000 2021-03-09T05:32:23.084 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:32:24.883 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T05:32:24.883 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:32:24.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:32:24.884 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T05:32:24.885 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T05:32:24.885 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T05:32:24.888 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T05:32:24.888 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T05:32:54.742 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T05:32:54.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T05:32:54.744 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T05:32:54.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T05:32:54.738+0000 7f2039ac0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 45174) UID: 1000 2021-03-09T05:32:54.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:32:56.745 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T05:32:56.746 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:32:56.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:32:56.746 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T05:32:56.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T05:32:56.747 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T05:32:56.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T05:32:56.750 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T05:33:26.004 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:33:26.002+0000 7f901a51c700 -1 reset not still connected to 0x5588f96f1450 2021-03-09T05:33:26.005 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:33:26.006+0000 7f901a51c700 -1 reset not still connected to 0x5588f96f3cf0 2021-03-09T05:33:26.006 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:33:26.006+0000 7f901a51c700 -1 reset not still connected to 0x558903e2bd40 2021-03-09T05:33:26.006 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:33:26.006+0000 7f901a51c700 -1 reset not still connected to 0x558903e2be10 2021-03-09T05:33:33.889 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6485768615065658) > weight (0.25) 2021-03-09T05:33:33.891 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T05:33:33.892 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T05:33:33.892 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T05:33:33.893 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T05:33:33.885+0000 7fbfb59ca700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 43831) UID: 1000 2021-03-09T05:33:34.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:33:34.894 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T05:33:34.894 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:33:34.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:33:34.895 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T05:33:34.895 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T05:33:34.896 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T05:33:34.898 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T05:33:34.899 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T05:34:02.752 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6443595199408423) > weight (0.25) 2021-03-09T05:34:02.754 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:34:02.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:34:02.755 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T05:34:02.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:34:02.753+0000 7f00ce5df700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 44981) UID: 1000 2021-03-09T05:34:02.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:34:05.756 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:34:05.756 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:34:05.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:34:05.757 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:34:05.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:34:05.758 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:34:05.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:34:05.761 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T05:35:15.761 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8602310335028175) > weight (0.25) 2021-03-09T05:35:15.767 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:35:15.768 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:35:15.768 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T05:35:15.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:35:15.764+0000 7f667da11700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 45646) UID: 1000 2021-03-09T05:35:15.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:35:17.769 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:35:17.770 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:35:17.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:35:17.771 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:35:17.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:35:17.771 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:35:17.774 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:35:17.775 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T05:35:30.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6277313982476886) > weight (0.25) 2021-03-09T05:35:30.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6475508120728668) > weight (0.25) 2021-03-09T05:35:30.901 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5827423604034385) > weight (0.25) 2021-03-09T05:35:30.901 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5239156032610628) > weight (0.25) 2021-03-09T05:35:30.901 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-03-09T05:36:17.776 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4472203448169385) > weight (0.25) 2021-03-09T05:36:17.778 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5968208182383513) > weight (0.25) 2021-03-09T05:36:17.778 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9965725488746398) > weight (0.25) 2021-03-09T05:36:17.778 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T05:36:17.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T05:36:17.779 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T05:36:17.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T05:36:17.775+0000 7f41dd25b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 43051) UID: 1000 2021-03-09T05:36:17.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:36:20.780 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T05:36:20.780 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:36:20.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:36:20.781 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T05:36:20.782 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T05:36:20.782 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T05:36:20.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T05:36:20.786 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-03-09T05:36:45.902 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T05:36:45.905 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T05:36:45.906 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T05:36:45.909 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T05:36:45.906+0000 7faa55650700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 45365) UID: 1000 2021-03-09T05:36:46.110 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:36:47.907 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T05:36:47.908 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:36:47.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:36:47.908 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T05:36:47.909 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T05:36:47.909 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T05:36:47.912 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T05:36:47.912 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T05:36:51.077 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:36:51.072+0000 7f901a51c700 -1 reset not still connected to 0x5588f2550cb0 2021-03-09T05:36:51.078 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:36:51.076+0000 7f901a51c700 -1 reset not still connected to 0x558903e2b930 2021-03-09T05:37:38.787 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T05:37:38.790 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T05:37:38.791 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T05:37:38.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T05:37:38.789+0000 7f5227f25700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 45456) UID: 1000 2021-03-09T05:37:39.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:37:45.792 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T05:37:45.793 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:37:45.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:37:45.794 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T05:37:45.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T05:37:45.795 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T05:37:45.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T05:37:45.798 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T05:38:16.914 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8008353182418004) > weight (0.25) 2021-03-09T05:38:16.916 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T05:38:16.917 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T05:38:16.918 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T05:38:16.920 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T05:38:16.913+0000 7f5996224700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 45555) UID: 1000 2021-03-09T05:38:17.119 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:38:23.919 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T05:38:23.920 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:38:23.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:38:23.921 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T05:38:23.921 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T05:38:23.921 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T05:38:23.924 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T05:38:23.924 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T05:38:48.161 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:38:48.156+0000 7f082a074700 -1 reset not still connected to 0x55a9911245b0 2021-03-09T05:38:48.161 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:38:48.156+0000 7f082a074700 -1 reset not still connected to 0x55a99d0547d0 2021-03-09T05:38:48.162 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:38:48.156+0000 7f082a074700 -1 reset not still connected to 0x55a9f121c080 2021-03-09T05:38:48.162 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:38:48.156+0000 7f082a074700 -1 reset not still connected to 0x55aa12b20700 2021-03-09T05:39:02.799 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3618677088715131) > weight (0.25) 2021-03-09T05:39:02.801 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:39:02.802 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:39:02.803 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T05:39:02.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:39:02.800+0000 7f62c3bf5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 45749) UID: 1000 2021-03-09T05:39:03.003 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:39:03.804 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:39:03.804 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:39:03.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:39:03.805 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:39:03.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:39:03.806 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:39:03.813 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:39:03.813 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T05:39:57.926 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.27501559246558516) > weight (0.25) 2021-03-09T05:39:57.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7777708873904328) > weight (0.25) 2021-03-09T05:39:57.929 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4337722050161883) > weight (0.25) 2021-03-09T05:39:57.930 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8988180111193191) > weight (0.25) 2021-03-09T05:39:57.930 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T05:40:09.814 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.782425878664511) > weight (0.25) 2021-03-09T05:40:09.818 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8021316180380543) > weight (0.25) 2021-03-09T05:40:09.819 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T05:40:09.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T05:40:09.820 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T05:40:09.821 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T05:40:09.819+0000 7f3649304700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 43244) UID: 1000 2021-03-09T05:40:10.021 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:40:10.821 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T05:40:10.821 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:40:10.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:40:10.823 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T05:40:10.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T05:40:10.824 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T05:40:10.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T05:40:10.827 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T05:41:04.932 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8946319328647514) > weight (0.25) 2021-03-09T05:41:04.933 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T05:41:04.934 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T05:41:04.935 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T05:41:04.936 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T05:41:04.930+0000 7f3134f95700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 46139) UID: 1000 2021-03-09T05:41:05.136 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:41:12.935 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T05:41:12.936 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:41:12.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:41:12.937 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T05:41:12.937 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T05:41:12.937 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T05:41:12.940 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T05:41:12.941 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T05:41:26.829 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3370661049400173) > weight (0.25) 2021-03-09T05:41:26.830 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:41:26.831 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:41:26.832 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T05:41:26.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:41:26.826+0000 7f995706e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 46231) UID: 1000 2021-03-09T05:41:27.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:41:28.833 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:41:28.834 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:41:28.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:41:28.835 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:41:28.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:41:28.836 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:41:28.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:41:28.839 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T05:42:37.942 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9810892776864679) > weight (0.25) 2021-03-09T05:42:37.943 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9822469344554143) > weight (0.25) 2021-03-09T05:42:37.944 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T05:42:37.945 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T05:42:37.945 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T05:42:37.946 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T05:42:37.936+0000 7fb5ff6fc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 45273) UID: 1000 2021-03-09T05:42:38.146 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:42:46.946 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T05:42:46.949 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:42:46.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:42:46.950 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T05:42:46.950 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T05:42:46.950 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T05:42:46.953 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T05:42:46.954 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T05:43:08.841 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5232004033663683) > weight (0.25) 2021-03-09T05:43:08.841 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:43:08.842 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:43:08.842 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T05:43:08.843 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:43:08.843 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:43:08.844 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:43:08.840+0000 7fdcef13b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 46522) UID: 1000 2021-03-09T05:43:09.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:43:14.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:43:14.946 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:43:14.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:43:14.946 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:43:14.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:43:14.949 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T05:44:16.950 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.990486725122808) > weight (0.25) 2021-03-09T05:44:16.951 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:44:16.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:44:16.952 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T05:44:16.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:44:16.947+0000 7fea8525d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 46714) UID: 1000 2021-03-09T05:44:17.153 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:44:17.953 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:44:17.953 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:44:17.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:44:17.954 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:44:17.954 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:44:17.955 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:44:17.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:44:17.958 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T05:44:27.954 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5797757445484449) > weight (0.25) 2021-03-09T05:44:27.955 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T05:44:27.955 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T05:44:27.956 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T05:44:27.957 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T05:44:27.955+0000 7f3c2c556700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 46431) UID: 1000 2021-03-09T05:44:28.157 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:44:33.956 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T05:44:33.957 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:44:33.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:44:33.958 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T05:44:33.958 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T05:44:33.958 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T05:44:33.969 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T05:44:33.969 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T05:45:41.959 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T05:45:41.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T05:45:41.961 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T05:45:41.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T05:45:41.953+0000 7fd598b27700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 46040) UID: 1000 2021-03-09T05:45:42.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:45:43.341 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:45:43.332+0000 7f901a51c700 -1 reset not still connected to 0x5588e7f44cb0 2021-03-09T05:45:43.342 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:45:43.332+0000 7f901a51c700 -1 reset not still connected to 0x5588fa398750 2021-03-09T05:45:43.342 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:45:43.332+0000 7f901a51c700 -1 reset not still connected to 0x5589045915f0 2021-03-09T05:45:47.962 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T05:45:47.962 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:45:47.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:45:47.963 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T05:45:47.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T05:45:47.964 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T05:45:47.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T05:45:47.967 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-03-09T05:45:58.469 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T05:45:58.461+0000 7fc0e0d45700 -1 reset not still connected to 0x55e9194fcc30 2021-03-09T05:46:28.970 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5416017873719886) > weight (0.25) 2021-03-09T05:46:28.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.27159292739587926) > weight (0.25) 2021-03-09T05:46:28.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4680312076941434) > weight (0.25) 2021-03-09T05:46:28.971 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T05:46:28.972 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T05:46:28.972 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T05:46:28.973 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T05:46:28.973 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:46:28.974 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T05:46:28.969+0000 7f3c5b47a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 44024) UID: 1000 2021-03-09T05:46:29.174 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:46:35.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:46:35.075 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T05:46:35.076 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T05:46:35.076 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T05:46:35.080 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T05:46:35.080 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-03-09T05:46:58.968 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8981376579501468) > weight (0.25) 2021-03-09T05:46:58.969 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T05:46:58.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T05:46:58.971 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T05:46:58.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T05:46:58.968+0000 7f33ba645700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 46881) UID: 1000 2021-03-09T05:46:59.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:47:01.971 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T05:47:01.972 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:47:01.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:47:01.973 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T05:47:01.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T05:47:01.974 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T05:47:01.977 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T05:47:01.977 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T05:47:43.081 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38778028161807765) > weight (0.25) 2021-03-09T05:47:43.085 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6179498941345704) > weight (0.25) 2021-03-09T05:47:43.085 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.617025035712354) > weight (0.25) 2021-03-09T05:47:43.085 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6461667477783979) > weight (0.25) 2021-03-09T05:47:43.086 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T05:48:48.979 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T05:48:48.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T05:48:48.981 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T05:48:48.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T05:48:48.978+0000 7fcc86f0f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 47075) UID: 1000 2021-03-09T05:48:49.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:48:53.982 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T05:48:53.983 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:48:53.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:48:53.984 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T05:48:53.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T05:48:53.984 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T05:48:53.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T05:48:53.998 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T05:48:59.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7082666716805427) > weight (0.25) 2021-03-09T05:48:59.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2619384908734561) > weight (0.25) 2021-03-09T05:48:59.088 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.37865873836067176) > weight (0.25) 2021-03-09T05:48:59.088 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.748331451177344) > weight (0.25) 2021-03-09T05:48:59.088 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T05:50:09.089 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T05:50:09.098 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T05:50:09.099 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T05:50:09.100 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T05:50:09.093+0000 7f1a5012b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 45948) UID: 1000 2021-03-09T05:50:09.300 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:50:16.100 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T05:50:16.103 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:50:16.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:50:16.104 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T05:50:16.104 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T05:50:16.105 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T05:50:16.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T05:50:16.109 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T05:50:16.999 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.51663096727475) > weight (0.25) 2021-03-09T05:50:17.000 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.44596631121829533) > weight (0.25) 2021-03-09T05:50:17.000 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.43083877716181174) > weight (0.25) 2021-03-09T05:50:17.001 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7599568929994132) > weight (0.25) 2021-03-09T05:50:17.001 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-03-09T05:50:42.312 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T05:50:42.305+0000 7f7861cbd700 -1 rbd::mirror::ImageReplayer: 0x55962e38b980 [3/bd4ef0c3-a3a7-42f2-8034-c828cab83717] start: already running 2021-03-09T05:51:40.110 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.47966671109965797) > weight (0.25) 2021-03-09T05:51:40.112 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6864878894224423) > weight (0.25) 2021-03-09T05:51:40.112 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.509206541900819) > weight (0.25) 2021-03-09T05:51:40.113 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.77565989672053) > weight (0.25) 2021-03-09T05:51:40.113 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T05:51:55.002 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6738835326203257) > weight (0.25) 2021-03-09T05:51:55.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.278000380625698) > weight (0.25) 2021-03-09T05:51:55.008 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.537233707100723) > weight (0.25) 2021-03-09T05:51:55.009 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9219567685396178) > weight (0.25) 2021-03-09T05:51:55.009 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T05:53:10.114 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8116345098003738) > weight (0.25) 2021-03-09T05:53:10.150 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5548791163372286) > weight (0.25) 2021-03-09T05:53:10.150 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5819616656424762) > weight (0.25) 2021-03-09T05:53:10.151 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T05:53:10.152 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T05:53:10.152 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T05:53:10.153 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T05:53:10.150+0000 7f87bce9a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 47175) UID: 1000 2021-03-09T05:53:10.354 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:53:13.154 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T05:53:13.154 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:53:13.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:53:13.156 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T05:53:13.156 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T05:53:13.157 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T05:53:13.161 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T05:53:13.161 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-03-09T05:53:42.010 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.260811695244641) > weight (0.25) 2021-03-09T05:53:42.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5816707949390542) > weight (0.25) 2021-03-09T05:53:42.013 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8192886224726125) > weight (0.25) 2021-03-09T05:53:42.014 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.397857531475191) > weight (0.25) 2021-03-09T05:53:42.014 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T05:54:51.162 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9581746678790398) > weight (0.25) 2021-03-09T05:54:51.165 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6360698651130042) > weight (0.25) 2021-03-09T05:54:51.166 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6933389346979191) > weight (0.25) 2021-03-09T05:54:51.166 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T05:54:51.167 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T05:54:51.167 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T05:54:51.168 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T05:54:51.164+0000 7f1fbc36a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 47560) UID: 1000 2021-03-09T05:54:51.368 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:54:53.168 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T05:54:53.168 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:54:53.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:54:53.169 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T05:54:53.169 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T05:54:53.170 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T05:54:53.172 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T05:54:53.173 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T05:55:23.405 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T05:55:23.399+0000 7f901a51c700 -1 reset not still connected to 0x5588f96f2150 2021-03-09T05:55:27.015 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T05:55:27.018 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T05:55:27.019 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T05:55:27.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T05:55:27.016+0000 7f8bce90f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 47367) UID: 1000 2021-03-09T05:55:27.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:55:34.019 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T05:55:34.021 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:55:34.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:55:34.021 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T05:55:34.022 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T05:55:34.022 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T05:55:34.025 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T05:55:34.025 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T05:56:03.174 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9458185130052715) > weight (0.25) 2021-03-09T05:56:03.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.37082499897447885) > weight (0.25) 2021-03-09T05:56:03.175 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T05:56:03.176 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T05:56:03.177 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T05:56:03.178 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T05:56:03.175+0000 7f78624be700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 46622) UID: 1000 2021-03-09T05:56:03.378 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:56:10.178 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T05:56:10.180 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:56:10.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:56:10.181 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T05:56:10.182 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T05:56:10.182 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T05:56:10.186 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T05:56:10.187 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T05:57:27.188 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2648433865806097) > weight (0.25) 2021-03-09T05:57:27.189 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9840833712957066) > weight (0.25) 2021-03-09T05:57:27.189 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T05:57:27.190 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T05:57:27.190 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T05:57:27.190 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T05:57:27.191 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:57:27.191 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T05:57:27.186+0000 7fa7069b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 47845) UID: 1000 2021-03-09T05:57:27.391 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:57:29.026 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T05:57:29.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T05:57:29.027 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T05:57:29.028 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T05:57:29.028 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:57:29.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T05:57:29.026+0000 7f42d2541700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 47745) UID: 1000 2021-03-09T05:57:29.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:57:33.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:57:33.292 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T05:57:33.293 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T05:57:33.293 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T05:57:33.296 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T05:57:33.296 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T05:57:35.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:57:35.130 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T05:57:35.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T05:57:35.131 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T05:57:35.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T05:57:35.134 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T05:58:40.297 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5899217652503369) > weight (0.25) 2021-03-09T05:58:40.298 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.39979336847093705) > weight (0.25) 2021-03-09T05:58:40.299 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T05:58:40.299 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T05:58:40.300 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T05:58:40.301 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T05:58:40.293+0000 7f75c53d9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 47937) UID: 1000 2021-03-09T05:58:40.501 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T05:58:43.300 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T05:58:43.301 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T05:58:43.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T05:58:43.302 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T05:58:43.302 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T05:58:43.302 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T05:58:43.308 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T05:58:43.309 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T05:59:11.499 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:59:11.496+0000 7f082a074700 -1 reset not still connected to 0x55a97b99fee0 2021-03-09T05:59:11.499 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:59:11.496+0000 7f082a074700 -1 reset not still connected to 0x55a9935b4270 2021-03-09T05:59:11.499 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:59:11.496+0000 7f082a074700 -1 reset not still connected to 0x55a99d053e10 2021-03-09T05:59:19.135 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9074494659304474) > weight (0.25) 2021-03-09T05:59:19.136 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.450617476430348) > weight (0.25) 2021-03-09T05:59:19.136 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4120311327873726) > weight (0.25) 2021-03-09T05:59:19.136 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2992156991562367) > weight (0.25) 2021-03-09T05:59:19.137 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T05:59:46.346 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:59:46.340+0000 7f082a074700 -1 reset not still connected to 0x55a99f964630 2021-03-09T05:59:46.346 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T05:59:46.340+0000 7f082a074700 -1 reset not still connected to 0x55a99f9648a0 2021-03-09T06:00:24.310 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T06:00:24.312 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T06:00:24.312 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T06:00:24.315 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T06:00:24.311+0000 7fabecafb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 47467) UID: 1000 2021-03-09T06:00:24.516 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:00:31.313 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T06:00:31.322 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:00:31.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:00:31.323 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T06:00:31.323 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T06:00:31.323 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T06:00:31.326 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T06:00:31.327 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-03-09T06:01:15.138 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2954160308996032) > weight (0.25) 2021-03-09T06:01:15.142 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5569294818714914) > weight (0.25) 2021-03-09T06:01:15.143 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9074700816191001) > weight (0.25) 2021-03-09T06:01:15.143 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.43820503696208646) > weight (0.25) 2021-03-09T06:01:15.143 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T06:02:07.328 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T06:02:07.365 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T06:02:07.366 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T06:02:07.367 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T06:02:07.361+0000 7f81491fe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 48220) UID: 1000 2021-03-09T06:02:07.567 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:02:16.368 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T06:02:16.371 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:02:16.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:02:16.372 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T06:02:16.372 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T06:02:16.376 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T06:02:16.381 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T06:02:16.382 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T06:03:01.145 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T06:03:01.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T06:03:01.151 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T06:03:01.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T06:03:01.148+0000 7fac31e99700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 48028) UID: 1000 2021-03-09T06:03:01.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:03:09.152 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T06:03:09.153 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:03:09.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:03:09.154 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T06:03:09.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T06:03:09.155 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T06:03:09.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T06:03:09.159 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-03-09T06:03:56.383 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T06:03:56.387 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T06:03:56.388 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T06:03:56.389 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T06:03:56.383+0000 7f0f9b572700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 48314) UID: 1000 2021-03-09T06:03:56.589 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:04:04.388 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T06:04:04.394 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:04:04.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:04:04.395 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T06:04:04.395 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T06:04:04.396 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T06:04:04.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T06:04:04.399 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T06:05:07.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.29186874844892907) > weight (0.25) 2021-03-09T06:05:07.162 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45904702166468925) > weight (0.25) 2021-03-09T06:05:07.162 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7005509645024653) > weight (0.25) 2021-03-09T06:05:07.162 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6145882054918428) > weight (0.25) 2021-03-09T06:05:07.162 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T06:05:13.401 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.446654078773531) > weight (0.25) 2021-03-09T06:05:13.407 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.45088710849778457) > weight (0.25) 2021-03-09T06:05:13.407 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T06:05:13.408 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T06:05:13.408 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T06:05:13.409 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T06:05:13.402+0000 7f97a35ab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 48128) UID: 1000 2021-03-09T06:05:13.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:05:20.409 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T06:05:20.411 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:05:20.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:05:20.412 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T06:05:20.412 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T06:05:20.415 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T06:05:20.419 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T06:05:20.420 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T06:06:19.164 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.46790142144571745) > weight (0.25) 2021-03-09T06:06:19.167 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.860060930312563) > weight (0.25) 2021-03-09T06:06:19.167 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7323453157515595) > weight (0.25) 2021-03-09T06:06:19.167 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5752526107263675) > weight (0.25) 2021-03-09T06:06:19.168 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T06:06:49.421 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6138133903414383) > weight (0.25) 2021-03-09T06:06:49.423 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3722546720243568) > weight (0.25) 2021-03-09T06:06:49.423 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5833371510491159) > weight (0.25) 2021-03-09T06:06:49.423 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4470389665027863) > weight (0.25) 2021-03-09T06:06:49.423 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T06:07:21.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9729062615103651) > weight (0.25) 2021-03-09T06:07:21.171 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.46284752224063397) > weight (0.25) 2021-03-09T06:07:21.171 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6971415335544324) > weight (0.25) 2021-03-09T06:07:21.171 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8443566445164328) > weight (0.25) 2021-03-09T06:07:21.171 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T06:07:59.425 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8659139787556775) > weight (0.25) 2021-03-09T06:07:59.427 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8346955905018992) > weight (0.25) 2021-03-09T06:07:59.427 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T06:07:59.428 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T06:07:59.428 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T06:07:59.428 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T06:07:59.429 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:07:59.430 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T06:07:59.423+0000 7f8de7dc1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 48601) UID: 1000 2021-03-09T06:07:59.629 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:08:05.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:08:05.531 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T06:08:05.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T06:08:05.532 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T06:08:05.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T06:08:05.535 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T06:09:02.181 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5246299799124278) > weight (0.25) 2021-03-09T06:09:02.184 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3610117325407004) > weight (0.25) 2021-03-09T06:09:02.184 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7537713731294641) > weight (0.25) 2021-03-09T06:09:02.184 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9083523921537147) > weight (0.25) 2021-03-09T06:09:02.185 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T06:09:19.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.486781490059514) > weight (0.25) 2021-03-09T06:09:19.538 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3543734071780531) > weight (0.25) 2021-03-09T06:09:19.538 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.26615645799532117) > weight (0.25) 2021-03-09T06:09:19.539 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8971064853730574) > weight (0.25) 2021-03-09T06:09:19.539 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T06:10:06.185 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7552034329269172) > weight (0.25) 2021-03-09T06:10:06.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9607897239923269) > weight (0.25) 2021-03-09T06:10:06.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.30482019088081735) > weight (0.25) 2021-03-09T06:10:06.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5742632756230449) > weight (0.25) 2021-03-09T06:10:06.279 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T06:10:26.540 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7038319887583737) > weight (0.25) 2021-03-09T06:10:26.541 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9842805670519217) > weight (0.25) 2021-03-09T06:10:26.542 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T06:10:26.543 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T06:10:26.543 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T06:10:26.544 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T06:10:26.541+0000 7f1beee68700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 48693) UID: 1000 2021-03-09T06:10:26.745 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:10:32.544 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T06:10:32.545 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:10:32.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:10:32.545 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T06:10:32.546 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T06:10:32.546 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T06:10:32.549 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T06:10:32.549 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T06:11:18.280 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8889006200064477) > weight (0.25) 2021-03-09T06:11:18.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8633946765708916) > weight (0.25) 2021-03-09T06:11:18.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7177628909317553) > weight (0.25) 2021-03-09T06:11:18.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3260497921944435) > weight (0.25) 2021-03-09T06:11:18.283 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T06:12:00.550 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T06:12:00.554 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T06:12:00.554 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T06:12:00.555 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T06:12:00.551+0000 7f42873cf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 48507) UID: 1000 2021-03-09T06:12:00.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:12:09.556 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T06:12:09.557 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:12:09.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:12:09.559 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T06:12:09.559 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T06:12:09.560 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T06:12:09.564 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T06:12:09.565 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T06:12:41.284 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.388980845527259) > weight (0.25) 2021-03-09T06:12:41.287 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4621760784675023) > weight (0.25) 2021-03-09T06:12:41.288 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T06:12:41.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T06:12:41.289 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T06:12:41.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T06:12:41.287+0000 7feea9f8f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 46331) UID: 1000 2021-03-09T06:12:41.490 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:12:43.290 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T06:12:43.290 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:12:43.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:12:43.291 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T06:12:43.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T06:12:43.292 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T06:12:43.314 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T06:12:43.314 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T06:13:54.566 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8514250512150362) > weight (0.25) 2021-03-09T06:13:54.569 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7867906006749118) > weight (0.25) 2021-03-09T06:13:54.569 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5007119230990177) > weight (0.25) 2021-03-09T06:13:54.569 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T06:13:54.570 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T06:13:54.571 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T06:13:54.571 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T06:13:54.565+0000 7fd91a554700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 47651) UID: 1000 2021-03-09T06:13:54.772 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:13:55.316 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8048845769983481) > weight (0.25) 2021-03-09T06:13:55.316 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7587107800791186) > weight (0.25) 2021-03-09T06:13:55.316 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8608218229258593) > weight (0.25) 2021-03-09T06:13:55.317 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9864797803587103) > weight (0.25) 2021-03-09T06:13:55.317 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T06:13:58.572 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T06:13:58.572 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:13:58.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:13:58.573 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T06:13:58.574 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T06:13:58.574 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T06:13:58.585 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T06:13:58.585 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T06:14:56.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5847476165910267) > weight (0.25) 2021-03-09T06:14:56.351 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7460400802688697) > weight (0.25) 2021-03-09T06:14:56.351 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8727312644145994) > weight (0.25) 2021-03-09T06:14:56.351 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6790110775003427) > weight (0.25) 2021-03-09T06:14:56.352 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T06:15:54.586 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9335828356766261) > weight (0.25) 2021-03-09T06:15:54.590 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5014481779518188) > weight (0.25) 2021-03-09T06:15:54.590 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.625128077040417) > weight (0.25) 2021-03-09T06:15:54.591 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T06:15:54.592 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T06:15:54.592 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T06:15:54.593 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T06:15:54.593 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:15:54.600 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T06:15:54.591+0000 7fb7b4646700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 49070) UID: 1000 2021-03-09T06:15:54.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:16:00.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:16:00.696 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T06:16:00.696 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T06:16:00.697 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T06:16:00.700 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T06:16:00.700 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T06:16:19.837 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:16:19.831+0000 7f082a074700 -1 reset not still connected to 0x55a99db96c30 2021-03-09T06:16:19.838 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:16:19.831+0000 7f082a074700 -1 reset not still connected to 0x55a99db99c20 2021-03-09T06:16:19.838 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:16:19.831+0000 7f082a074700 -1 reset not still connected to 0x55a9f121c3c0 2021-03-09T06:16:39.353 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6744718965217427) > weight (0.25) 2021-03-09T06:16:39.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.39412159784556444) > weight (0.25) 2021-03-09T06:16:39.356 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7971663687219256) > weight (0.25) 2021-03-09T06:16:39.357 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4532772555113094) > weight (0.25) 2021-03-09T06:16:39.357 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T06:17:12.701 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T06:17:12.705 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T06:17:12.707 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T06:17:12.708 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T06:17:12.702+0000 7fc3bdc64700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 48879) UID: 1000 2021-03-09T06:17:12.907 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:17:16.708 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T06:17:16.708 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:17:16.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:17:16.709 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T06:17:16.709 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T06:17:16.710 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T06:17:16.713 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T06:17:16.714 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T06:17:46.358 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7719399859959393) > weight (0.25) 2021-03-09T06:17:46.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.47774828541612124) > weight (0.25) 2021-03-09T06:17:46.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4442728247975666) > weight (0.25) 2021-03-09T06:17:46.360 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4008073071629108) > weight (0.25) 2021-03-09T06:17:46.361 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T06:18:20.715 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6913192546848269) > weight (0.25) 2021-03-09T06:18:20.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9338860884280584) > weight (0.25) 2021-03-09T06:18:20.717 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T06:18:20.718 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T06:18:20.718 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T06:18:20.719 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T06:18:20.719 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:18:20.720 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T06:18:20.717+0000 7f88f86db700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 48787) UID: 1000 2021-03-09T06:18:20.919 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:18:26.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:18:26.824 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T06:18:26.824 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T06:18:26.824 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T06:18:26.831 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T06:18:26.831 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T06:18:53.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9113919021511622) > weight (0.25) 2021-03-09T06:18:53.365 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3725334862862568) > weight (0.25) 2021-03-09T06:18:53.365 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.713766440032466) > weight (0.25) 2021-03-09T06:18:53.365 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.41709285837679266) > weight (0.25) 2021-03-09T06:18:53.365 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T06:20:18.833 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.47553224151229245) > weight (0.25) 2021-03-09T06:20:18.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8113285541459206) > weight (0.25) 2021-03-09T06:20:18.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7177779539900925) > weight (0.25) 2021-03-09T06:20:18.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6129148472131727) > weight (0.25) 2021-03-09T06:20:18.838 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-03-09T06:20:52.366 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.29823709691569733) > weight (0.25) 2021-03-09T06:20:52.397 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5090117248853406) > weight (0.25) 2021-03-09T06:20:52.398 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7594132395819898) > weight (0.25) 2021-03-09T06:20:52.398 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2830967077973209) > weight (0.25) 2021-03-09T06:20:52.398 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T06:21:56.839 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4826323667898069) > weight (0.25) 2021-03-09T06:21:56.841 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5407725412287188) > weight (0.25) 2021-03-09T06:21:56.842 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.476352411209338) > weight (0.25) 2021-03-09T06:21:56.842 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T06:21:56.843 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T06:21:56.844 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T06:21:56.845 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T06:21:56.837+0000 7fdcc9956700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 49166) UID: 1000 2021-03-09T06:21:57.045 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:21:58.052 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:21:58.049+0000 7f901a51c700 -1 reset not still connected to 0x558903e2add0 2021-03-09T06:21:58.122 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:21:58.117+0000 7f082a074700 -1 reset not still connected to 0x55a9968a8f70 2021-03-09T06:21:58.122 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:21:58.117+0000 7f082a074700 -1 reset not still connected to 0x55a9968ab670 2021-03-09T06:21:58.122 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:21:58.117+0000 7f082a074700 -1 reset not still connected to 0x55a9975a2ff0 2021-03-09T06:22:02.845 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T06:22:02.845 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:22:02.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:22:02.846 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T06:22:02.846 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T06:22:02.847 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T06:22:02.850 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T06:22:02.850 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T06:22:19.399 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6018553929316467) > weight (0.25) 2021-03-09T06:22:19.399 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T06:22:19.400 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T06:22:19.400 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T06:22:19.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:22:19.397+0000 7f6c0e456700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 47267) UID: 1000 2021-03-09T06:22:19.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:22:27.401 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T06:22:27.402 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:22:27.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:22:27.403 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T06:22:27.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T06:22:27.404 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T06:22:27.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T06:22:27.409 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T06:23:32.410 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6418249939162056) > weight (0.25) 2021-03-09T06:23:32.412 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8477489583884249) > weight (0.25) 2021-03-09T06:23:32.412 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5780186313190765) > weight (0.25) 2021-03-09T06:23:32.413 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9400658525110857) > weight (0.25) 2021-03-09T06:23:32.413 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T06:23:42.851 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.26703555848974014) > weight (0.25) 2021-03-09T06:23:42.853 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4658747763276163) > weight (0.25) 2021-03-09T06:23:42.853 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4853722729320692) > weight (0.25) 2021-03-09T06:23:42.854 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5167993692986831) > weight (0.25) 2021-03-09T06:23:42.854 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T06:24:38.414 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T06:24:38.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T06:24:38.417 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T06:24:38.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T06:24:38.414+0000 7ff6556ac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 48406) UID: 1000 2021-03-09T06:24:38.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:24:47.419 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T06:24:47.426 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:24:47.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:24:47.427 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T06:24:47.427 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T06:24:47.428 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T06:24:47.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T06:24:47.433 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T06:25:16.855 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5849949883988307) > weight (0.25) 2021-03-09T06:25:16.860 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.479088064070714) > weight (0.25) 2021-03-09T06:25:16.860 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T06:25:16.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T06:25:16.861 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T06:25:16.862 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T06:25:16.858+0000 7f00270fd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 49352) UID: 1000 2021-03-09T06:25:17.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:25:17.610 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:25:17.606+0000 7f082a074700 -1 reset not still connected to 0x55a98f9cf5a0 2021-03-09T06:25:17.612 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:25:17.606+0000 7f082a074700 -1 reset not still connected to 0x55a991127810 2021-03-09T06:25:17.612 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:25:17.606+0000 7f082a074700 -1 reset not still connected to 0x55a996c44be0 2021-03-09T06:25:17.613 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:25:17.606+0000 7f082a074700 -1 reset not still connected to 0x55a99db972b0 2021-03-09T06:25:17.613 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:25:17.606+0000 7f082a074700 -1 reset not still connected to 0x55a99db99740 2021-03-09T06:25:23.862 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T06:25:23.874 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:25:23.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:25:23.875 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T06:25:23.876 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T06:25:23.878 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T06:25:23.881 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T06:25:23.881 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T06:26:44.434 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9838184575799964) > weight (0.25) 2021-03-09T06:26:44.439 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9159537685336602) > weight (0.25) 2021-03-09T06:26:44.440 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T06:26:44.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T06:26:44.442 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T06:26:44.443 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T06:26:44.444 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:26:44.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T06:26:44.440+0000 7fa062391700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 48970) UID: 1000 2021-03-09T06:26:44.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:26:50.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:26:50.546 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T06:26:50.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T06:26:50.547 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T06:26:50.550 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T06:26:50.551 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T06:27:06.882 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9040966253035977) > weight (0.25) 2021-03-09T06:27:06.885 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T06:27:06.886 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T06:27:06.886 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T06:27:06.887 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T06:27:06.884+0000 7f1a5b630700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 46980) UID: 1000 2021-03-09T06:27:07.088 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:27:08.048 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:27:08.044+0000 7f082a074700 -1 reset not still connected to 0x55aa12b20cb0 2021-03-09T06:27:13.888 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T06:27:13.889 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:27:13.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:27:13.890 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T06:27:13.891 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T06:27:13.892 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T06:27:13.896 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T06:27:13.896 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T06:28:03.091 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:28:03.087+0000 7f901a51c700 -1 reset not still connected to 0x558903e2d400 2021-03-09T06:28:38.552 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T06:28:38.556 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T06:28:38.557 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T06:28:38.558 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T06:28:38.554+0000 7f8e75a69700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 49638) UID: 1000 2021-03-09T06:28:38.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:28:41.557 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T06:28:41.558 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:28:41.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:28:41.559 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T06:28:41.559 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T06:28:41.560 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T06:28:41.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T06:28:41.573 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T06:28:47.062 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T06:28:47.054+0000 7f12285f8700 -1 rbd::mirror::ImageReplayer: 0x55b011ad1180 [3/18a549b3-61f2-46e5-af4d-1a9b02c6e0ad] start: already running 2021-03-09T06:28:58.898 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8782943880072274) > weight (0.25) 2021-03-09T06:28:58.907 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8552822642927582) > weight (0.25) 2021-03-09T06:28:58.909 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8904252641835979) > weight (0.25) 2021-03-09T06:28:58.910 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.38024158238110894) > weight (0.25) 2021-03-09T06:28:58.911 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T06:30:33.574 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4065409196448754) > weight (0.25) 2021-03-09T06:30:33.608 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7194603154707118) > weight (0.25) 2021-03-09T06:30:33.608 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7647796632874834) > weight (0.25) 2021-03-09T06:30:33.608 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T06:30:33.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T06:30:33.610 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T06:30:33.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T06:30:33.608+0000 7ff30c3c1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 45849) UID: 1000 2021-03-09T06:30:34.011 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:30:35.007 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:30:35.006+0000 7f901a51c700 -1 reset not still connected to 0x5588fa39b740 2021-03-09T06:30:35.611 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T06:30:35.612 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:30:35.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:30:35.613 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T06:30:35.614 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T06:30:35.615 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T06:30:35.618 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T06:30:35.618 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T06:30:43.912 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6971807150158161) > weight (0.25) 2021-03-09T06:30:43.912 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T06:30:43.913 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T06:30:43.913 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T06:30:43.914 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T06:30:43.912+0000 7f5067687700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 50060) UID: 1000 2021-03-09T06:30:44.115 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:30:50.914 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T06:30:50.915 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:30:50.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:30:50.917 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T06:30:50.917 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T06:30:50.918 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T06:30:50.924 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T06:30:50.925 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T06:32:24.620 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9432237907099147) > weight (0.25) 2021-03-09T06:32:24.625 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T06:32:24.627 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T06:32:24.627 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T06:32:24.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:32:24.623+0000 7fcd3baf9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 49538) UID: 1000 2021-03-09T06:32:24.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:32:33.629 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T06:32:33.632 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:32:33.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:32:33.634 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T06:32:33.634 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T06:32:33.635 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T06:32:33.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T06:32:33.640 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T06:32:44.926 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.891982644242395) > weight (0.25) 2021-03-09T06:32:44.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3547680893549844) > weight (0.25) 2021-03-09T06:32:44.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8752020187120186) > weight (0.25) 2021-03-09T06:32:44.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6049264465241394) > weight (0.25) 2021-03-09T06:32:44.932 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-03-09T06:33:36.642 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T06:33:36.645 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T06:33:36.645 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T06:33:36.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T06:33:36.641+0000 7f1228df9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 50151) UID: 1000 2021-03-09T06:33:36.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:33:39.646 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T06:33:39.652 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:33:39.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:33:39.653 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T06:33:39.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T06:33:39.655 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T06:33:39.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T06:33:39.660 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T06:33:55.933 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8825720917654755) > weight (0.25) 2021-03-09T06:33:55.936 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.26403633123505676) > weight (0.25) 2021-03-09T06:33:55.936 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.46780424914957697) > weight (0.25) 2021-03-09T06:33:55.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.907550615266877) > weight (0.25) 2021-03-09T06:33:55.937 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T06:35:30.661 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9450910865987211) > weight (0.25) 2021-03-09T06:35:30.688 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5542192393080148) > weight (0.25) 2021-03-09T06:35:30.689 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7005785547576515) > weight (0.25) 2021-03-09T06:35:30.690 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5025132554181897) > weight (0.25) 2021-03-09T06:35:30.691 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-03-09T06:35:42.938 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7729517668663365) > weight (0.25) 2021-03-09T06:35:42.940 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T06:35:42.942 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T06:35:42.943 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T06:35:42.944 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T06:35:42.939+0000 7f37e78c9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 50352) UID: 1000 2021-03-09T06:35:43.144 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:35:48.944 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T06:35:48.946 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:35:48.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:35:48.948 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T06:35:48.948 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T06:35:48.951 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T06:35:48.961 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T06:35:48.962 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T06:36:41.693 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6916547446567388) > weight (0.25) 2021-03-09T06:36:41.694 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8897145268950807) > weight (0.25) 2021-03-09T06:36:41.695 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T06:36:41.696 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T06:36:41.697 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T06:36:41.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T06:36:41.694+0000 7f7c4ffda700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 49961) UID: 1000 2021-03-09T06:36:41.788 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T06:36:41.776+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad07496c0 2021-03-09T06:36:41.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:36:46.698 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T06:36:46.700 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:36:46.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:36:46.702 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T06:36:46.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T06:36:46.703 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T06:36:46.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T06:36:46.707 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T06:37:35.964 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8597075024514437) > weight (0.25) 2021-03-09T06:37:35.967 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7134121279591832) > weight (0.25) 2021-03-09T06:37:35.968 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8422326376117018) > weight (0.25) 2021-03-09T06:37:35.968 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4074353849877619) > weight (0.25) 2021-03-09T06:37:35.970 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T06:38:19.196 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:38:19.189+0000 7f082a074700 -1 reset not still connected to 0x55a99f965e90 2021-03-09T06:38:34.709 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6424977475509343) > weight (0.25) 2021-03-09T06:38:34.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.895254831216763) > weight (0.25) 2021-03-09T06:38:34.713 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.28107602410829624) > weight (0.25) 2021-03-09T06:38:34.713 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5806356347221712) > weight (0.25) 2021-03-09T06:38:34.714 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T06:38:39.970 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T06:38:39.972 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T06:38:39.972 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T06:38:39.979 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T06:38:39.976+0000 7f2bae11b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 49260) UID: 1000 2021-03-09T06:38:40.179 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:38:43.973 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T06:38:43.976 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:38:43.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:38:43.978 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T06:38:43.978 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T06:38:43.979 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T06:38:43.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T06:38:43.984 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T06:39:47.716 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2825833436948111) > weight (0.25) 2021-03-09T06:39:47.722 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5866796165629186) > weight (0.25) 2021-03-09T06:39:47.723 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4163905157989872) > weight (0.25) 2021-03-09T06:39:47.723 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T06:39:47.724 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T06:39:47.725 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T06:39:47.727 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T06:39:47.723+0000 7fd8b2aba700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 50252) UID: 1000 2021-03-09T06:39:47.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:39:48.726 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T06:39:48.727 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:39:48.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:39:48.728 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T06:39:48.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T06:39:48.728 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T06:39:48.732 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T06:39:48.733 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T06:39:53.986 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6979805187154425) > weight (0.25) 2021-03-09T06:39:53.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5525800472702662) > weight (0.25) 2021-03-09T06:39:53.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.609416788539216) > weight (0.25) 2021-03-09T06:39:53.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9389303279651917) > weight (0.25) 2021-03-09T06:39:53.993 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T06:40:54.734 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.39894287563988995) > weight (0.25) 2021-03-09T06:40:54.737 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T06:40:54.738 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T06:40:54.738 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T06:40:54.739 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:40:54.734+0000 7f2370097700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 50445) UID: 1000 2021-03-09T06:40:54.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:41:03.748 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T06:41:03.754 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:41:03.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:41:03.775 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T06:41:03.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T06:41:03.776 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T06:41:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T06:41:03.797 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T06:41:23.995 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6622859055533434) > weight (0.25) 2021-03-09T06:41:23.998 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T06:41:23.999 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T06:41:24.001 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T06:41:24.003 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T06:41:23.997+0000 7f88b03fc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 50646) UID: 1000 2021-03-09T06:41:24.200 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:41:28.003 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T06:41:28.003 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:41:28.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:41:28.005 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T06:41:28.005 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T06:41:28.006 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T06:41:28.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T06:41:28.021 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T06:42:25.799 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T06:42:25.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T06:42:25.802 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T06:42:25.802 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T06:42:25.802 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:42:25.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T06:42:25.800+0000 7fb458690700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 50544) UID: 1000 2021-03-09T06:42:26.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:42:31.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:42:31.907 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T06:42:31.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T06:42:31.908 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T06:42:31.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T06:42:31.912 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T06:42:40.022 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T06:42:40.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T06:42:40.025 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T06:42:40.026 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T06:42:40.024+0000 7f9b5abcc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 50839) UID: 1000 2021-03-09T06:42:40.226 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:42:48.026 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T06:42:48.028 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:42:48.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:42:48.029 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T06:42:48.029 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T06:42:48.031 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T06:42:48.033 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T06:42:48.033 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-03-09T06:43:59.035 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8450204856782423) > weight (0.25) 2021-03-09T06:43:59.068 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T06:43:59.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T06:43:59.070 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T06:43:59.076 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T06:43:59.071+0000 7f8525bd2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 51130) UID: 1000 2021-03-09T06:43:59.276 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:44:03.071 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T06:44:03.921 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:44:03.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:44:03.923 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T06:44:03.923 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T06:44:03.924 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T06:44:03.929 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T06:44:03.930 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T06:44:27.914 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4285522556025735) > weight (0.25) 2021-03-09T06:44:27.914 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5075947802303828) > weight (0.25) 2021-03-09T06:44:27.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4351041310655922) > weight (0.25) 2021-03-09T06:44:27.915 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T06:44:27.916 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T06:44:27.916 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T06:44:27.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T06:44:27.914+0000 7fb37ee97700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 50931) UID: 1000 2021-03-09T06:44:28.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:44:29.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:44:29.078+0000 7f8e69b20700 -1 rbd::mirror::ImageReplayer: 0x563c2a4d1e00 [3/7cda0e60-383d-4c8c-85dd-263c9aac913a] start: already running 2021-03-09T06:44:29.087 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:44:29.082+0000 7f8e6a321700 -1 rbd::mirror::ImageReplayer: 0x563c2a4d1180 [3/82811dfa-9edb-4d58-845c-054f2ae371f8] start: already running 2021-03-09T06:44:29.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:44:29.082+0000 7f8e6a321700 -1 rbd::mirror::ImageReplayer: 0x563c2a4d3480 [3/82fd07ce-ce87-4a9a-8826-1102d2c5803e] start: already running 2021-03-09T06:44:29.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:44:29.082+0000 7f8e6a321700 -1 rbd::mirror::ImageReplayer: 0x563c2a4d2f80 [3/941e8ddf-64f0-4f25-89c8-9cfbbec2992f] start: already running 2021-03-09T06:44:29.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:44:29.082+0000 7f8e6a321700 -1 rbd::mirror::ImageReplayer: 0x563c2a4d2a80 [3/c0bd3ade-03e9-4222-a37f-91aea372a1df] start: already running 2021-03-09T06:44:29.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:44:29.082+0000 7f8e6a321700 -1 rbd::mirror::ImageReplayer: 0x563c2a4d3200 [3/dc036c7f-f3e2-4032-b379-9d4600fb05ba] start: already running 2021-03-09T06:44:29.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:44:29.082+0000 7f8e6a321700 -1 rbd::mirror::ImageReplayer: 0x563c2a4d2d00 [3/fcee6884-fbbf-4b09-bfba-340ddeb5e81b] start: already running 2021-03-09T06:44:36.917 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T06:44:36.920 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:44:36.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:44:36.922 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T06:44:36.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T06:44:36.923 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T06:44:36.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T06:44:36.937 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T06:45:26.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9748816557176376) > weight (0.25) 2021-03-09T06:45:26.938 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8777069794131654) > weight (0.25) 2021-03-09T06:45:26.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8682566476106995) > weight (0.25) 2021-03-09T06:45:26.939 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6834540816396567) > weight (0.25) 2021-03-09T06:45:26.939 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T06:45:36.938 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7273201945249165) > weight (0.25) 2021-03-09T06:45:36.943 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T06:45:36.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T06:45:36.944 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T06:45:36.945 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T06:45:36.945 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:45:36.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:45:36.941+0000 7f8e6ab22700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 51031) UID: 1000 2021-03-09T06:45:37.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:45:43.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:45:43.047 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T06:45:43.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T06:45:43.048 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T06:45:43.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T06:45:43.053 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T06:46:26.940 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8653691299523918) > weight (0.25) 2021-03-09T06:46:27.013 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T06:46:27.014 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T06:46:27.015 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T06:46:27.016 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T06:46:27.012+0000 7f58bd147700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 51414) UID: 1000 2021-03-09T06:46:27.218 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:46:29.016 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T06:46:29.018 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:46:29.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:46:29.019 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T06:46:29.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T06:46:29.021 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T06:46:29.026 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T06:46:29.026 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T06:46:59.054 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T06:46:59.065 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T06:46:59.066 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T06:46:59.067 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T06:46:59.064+0000 7f16c24ac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 51222) UID: 1000 2021-03-09T06:46:59.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:47:07.067 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T06:47:07.076 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:47:07.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:47:07.077 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T06:47:07.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T06:47:07.080 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T06:47:07.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T06:47:07.084 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T06:48:26.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6989309125403025) > weight (0.25) 2021-03-09T06:48:26.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6733559062563187) > weight (0.25) 2021-03-09T06:48:26.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8037173006766996) > weight (0.25) 2021-03-09T06:48:26.077 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9407819711069323) > weight (0.25) 2021-03-09T06:48:26.078 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-03-09T06:48:41.085 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.42089792332177844) > weight (0.25) 2021-03-09T06:48:41.089 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8579097893495882) > weight (0.25) 2021-03-09T06:48:41.089 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T06:48:41.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T06:48:41.091 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T06:48:41.092 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T06:48:41.086+0000 7fb295bab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 50738) UID: 1000 2021-03-09T06:48:41.492 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:48:46.091 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T06:48:46.092 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:48:46.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:48:46.093 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T06:48:46.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T06:48:46.094 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T06:48:46.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T06:48:46.099 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T06:48:48.253 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:48:48.250+0000 7f082a074700 -1 reset not still connected to 0x55a9975a3810 2021-03-09T06:49:41.080 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5340513866557789) > weight (0.25) 2021-03-09T06:49:41.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7477778601977654) > weight (0.25) 2021-03-09T06:49:41.085 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5200756982164025) > weight (0.25) 2021-03-09T06:49:41.085 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2836642726304476) > weight (0.25) 2021-03-09T06:49:41.085 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T06:50:09.099 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5944488116090287) > weight (0.25) 2021-03-09T06:50:09.103 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4543112290151572) > weight (0.25) 2021-03-09T06:50:09.103 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6329513762082333) > weight (0.25) 2021-03-09T06:50:09.104 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T06:50:09.105 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T06:50:09.106 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T06:50:09.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T06:50:09.105+0000 7f9cc7b4a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 51506) UID: 1000 2021-03-09T06:50:09.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:50:10.482 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:50:10.477+0000 7f901a51c700 -1 reset not still connected to 0x5588fa399a00 2021-03-09T06:50:10.485 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:50:10.481+0000 7f901a51c700 -1 reset not still connected to 0x558903e2a410 2021-03-09T06:50:10.486 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:50:10.481+0000 7f901a51c700 -1 reset not still connected to 0x558904ab09c0 2021-03-09T06:50:10.486 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:50:10.481+0000 7f901a51c700 -1 reset not still connected to 0x558904ab1a00 2021-03-09T06:50:16.107 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T06:50:16.690 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:50:16.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:50:16.692 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T06:50:16.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T06:50:16.693 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T06:50:16.699 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T06:50:16.700 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T06:51:35.086 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5487345771389108) > weight (0.25) 2021-03-09T06:51:35.115 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T06:51:35.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T06:51:35.117 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T06:51:35.117 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T06:51:35.115+0000 7f7570f60700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 51707) UID: 1000 2021-03-09T06:51:35.318 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:51:43.118 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T06:51:43.119 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:51:43.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:51:43.120 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T06:51:43.121 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T06:51:43.121 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T06:51:43.124 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T06:51:43.124 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T06:51:44.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6446678358593382) > weight (0.25) 2021-03-09T06:51:44.701 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T06:51:44.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T06:51:44.702 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T06:51:44.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:51:44.699+0000 7f8ab05ed700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 51608) UID: 1000 2021-03-09T06:51:44.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:51:51.703 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T06:51:51.703 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:51:51.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:51:51.704 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T06:51:51.704 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T06:51:51.704 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T06:51:51.707 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T06:51:51.707 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T06:52:51.709 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.29237847457945076) > weight (0.25) 2021-03-09T06:52:51.714 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5194830281944479) > weight (0.25) 2021-03-09T06:52:51.715 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7332823953141053) > weight (0.25) 2021-03-09T06:52:51.715 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6802610539355306) > weight (0.25) 2021-03-09T06:52:51.715 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T06:53:38.125 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T06:53:38.160 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T06:53:38.161 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T06:53:38.161 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T06:53:38.161+0000 7fc249446700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 51321) UID: 1000 2021-03-09T06:53:38.362 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:53:43.349 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T06:53:43.341+0000 7f082a074700 -1 reset not still connected to 0x55a9a13abfb0 2021-03-09T06:53:46.162 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T06:53:46.162 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:53:46.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:53:46.163 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T06:53:46.164 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T06:53:46.165 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T06:53:46.177 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T06:53:46.178 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-03-09T06:54:25.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5439781989306833) > weight (0.25) 2021-03-09T06:54:25.719 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.43940610219106235) > weight (0.25) 2021-03-09T06:54:25.719 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3623418988960586) > weight (0.25) 2021-03-09T06:54:25.720 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9451576273853405) > weight (0.25) 2021-03-09T06:54:25.720 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T06:55:28.721 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6594685497712693) > weight (0.25) 2021-03-09T06:55:28.749 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T06:55:28.750 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T06:55:28.751 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T06:55:28.752 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:55:28.747+0000 7f6aa755c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 52267) UID: 1000 2021-03-09T06:55:28.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:55:29.751 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T06:55:29.752 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:55:29.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:55:29.753 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T06:55:29.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T06:55:29.753 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T06:55:29.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T06:55:29.764 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T06:55:44.178 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T06:55:44.180 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T06:55:44.180 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T06:55:44.181 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T06:55:44.179+0000 7f7e416a4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 52367) UID: 1000 2021-03-09T06:55:44.382 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:55:46.181 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T06:55:46.182 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:55:46.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:55:46.183 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T06:55:46.183 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T06:55:46.186 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T06:55:46.189 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T06:55:46.189 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-03-09T06:56:58.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7527965558201994) > weight (0.25) 2021-03-09T06:56:58.771 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T06:56:58.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T06:56:58.772 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T06:56:58.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:56:58.770+0000 7fae4d4c4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 52462) UID: 1000 2021-03-09T06:56:58.973 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:57:06.773 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T06:57:06.775 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:57:06.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:57:06.776 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T06:57:06.776 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T06:57:06.778 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T06:57:06.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T06:57:06.781 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T06:57:21.194 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3440306409493801) > weight (0.25) 2021-03-09T06:57:21.196 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8327371727395988) > weight (0.25) 2021-03-09T06:57:21.196 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T06:57:21.197 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T06:57:21.197 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T06:57:21.198 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T06:57:21.197+0000 7ff08b0d3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 49869) UID: 1000 2021-03-09T06:57:21.399 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:57:27.199 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T06:57:27.199 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:57:27.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:57:27.200 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T06:57:27.200 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T06:57:27.201 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T06:57:27.203 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T06:57:27.204 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T06:57:27.702 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:57:27.692+0000 7f901a51c700 -1 reset not still connected to 0x5588fa399a00 2021-03-09T06:57:27.702 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:57:27.696+0000 7f901a51c700 -1 reset not still connected to 0x5588fb947740 2021-03-09T06:57:27.703 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:57:27.696+0000 7f901a51c700 -1 reset not still connected to 0x558903e2a410 2021-03-09T06:57:27.703 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:57:27.696+0000 7f901a51c700 -1 reset not still connected to 0x558904ab09c0 2021-03-09T06:57:27.703 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T06:57:27.696+0000 7f901a51c700 -1 reset not still connected to 0x558904ab1a00 2021-03-09T06:58:55.205 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5858833577689256) > weight (0.25) 2021-03-09T06:58:55.218 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4319372499135371) > weight (0.25) 2021-03-09T06:58:55.218 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4077968895877876) > weight (0.25) 2021-03-09T06:58:55.219 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7120743739866688) > weight (0.25) 2021-03-09T06:58:55.219 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T06:59:03.783 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7775100240526388) > weight (0.25) 2021-03-09T06:59:03.785 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T06:59:03.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T06:59:03.786 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T06:59:03.786 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T06:59:03.787 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T06:59:03.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T06:59:03.784+0000 7fd72c69c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 52653) UID: 1000 2021-03-09T06:59:03.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T06:59:09.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T06:59:09.889 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T06:59:09.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T06:59:09.890 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T06:59:09.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T06:59:09.915 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T07:00:24.220 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6685680354538353) > weight (0.25) 2021-03-09T07:00:24.251 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T07:00:24.252 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T07:00:24.253 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T07:00:24.254 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T07:00:24.250+0000 7fc33caa3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 52101) UID: 1000 2021-03-09T07:00:24.454 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:00:31.254 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T07:00:31.257 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:00:31.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:00:31.258 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T07:00:31.258 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T07:00:31.259 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T07:00:31.261 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T07:00:31.262 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-03-09T07:01:00.915 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8541927620216607) > weight (0.25) 2021-03-09T07:01:00.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5051839335412418) > weight (0.25) 2021-03-09T07:01:00.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.928364773397234) > weight (0.25) 2021-03-09T07:01:00.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5116204422230081) > weight (0.25) 2021-03-09T07:01:00.919 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T07:02:06.263 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6647298528189773) > weight (0.25) 2021-03-09T07:02:06.287 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9920031046307555) > weight (0.25) 2021-03-09T07:02:06.288 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T07:02:06.290 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T07:02:06.291 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T07:02:06.292 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:02:06.288+0000 7f99a2c0f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 52752) UID: 1000 2021-03-09T07:02:06.492 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:02:07.292 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T07:02:07.293 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:02:07.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:02:07.293 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T07:02:07.294 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T07:02:07.294 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T07:02:07.297 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T07:02:07.298 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T07:02:24.921 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9524729872655003) > weight (0.25) 2021-03-09T07:02:24.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9486491675305129) > weight (0.25) 2021-03-09T07:02:24.924 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T07:02:24.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T07:02:24.925 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T07:02:24.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T07:02:24.924+0000 7f6807870700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 51898) UID: 1000 2021-03-09T07:02:25.126 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:02:30.926 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T07:02:30.927 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:02:30.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:02:30.928 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T07:02:30.928 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T07:02:30.929 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T07:02:30.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T07:02:30.934 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T07:02:36.804 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:02:36.800+0000 7f082a074700 -1 reset not still connected to 0x55a991fff400 2021-03-09T07:02:36.805 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:02:36.804+0000 7f082a074700 -1 reset not still connected to 0x55a996c425b0 2021-03-09T07:03:13.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7474715896239342) > weight (0.25) 2021-03-09T07:03:13.301 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.41053234672996275) > weight (0.25) 2021-03-09T07:03:13.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8050458683854836) > weight (0.25) 2021-03-09T07:03:13.302 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7552773963474397) > weight (0.25) 2021-03-09T07:03:13.302 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T07:04:22.942 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9817690895568146) > weight (0.25) 2021-03-09T07:04:22.981 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.30028992653987396) > weight (0.25) 2021-03-09T07:04:22.982 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T07:04:22.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T07:04:22.984 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T07:04:22.984 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T07:04:22.985 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:04:22.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T07:04:22.982+0000 7f41e3bdc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 53129) UID: 1000 2021-03-09T07:04:23.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:04:29.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:04:29.090 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T07:04:29.090 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T07:04:29.092 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T07:04:29.101 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T07:04:29.102 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T07:05:02.303 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T07:05:02.307 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T07:05:02.307 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T07:05:02.309 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T07:05:02.305+0000 7f581a084700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 52561) UID: 1000 2021-03-09T07:05:02.510 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:05:07.308 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T07:05:07.308 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:05:07.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:05:07.309 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T07:05:07.309 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T07:05:07.310 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T07:05:07.313 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T07:05:07.314 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T07:06:26.104 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:06:26.178 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:06:26.179 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T07:06:26.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:06:26.176+0000 7fc666e05700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 51799) UID: 1000 2021-03-09T07:06:26.380 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:06:33.181 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:06:33.181 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:06:33.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:06:33.183 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:06:33.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:06:33.183 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:06:33.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:06:33.187 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T07:06:46.315 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7757141198582537) > weight (0.25) 2021-03-09T07:06:46.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8742260947442152) > weight (0.25) 2021-03-09T07:06:46.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8030725058123703) > weight (0.25) 2021-03-09T07:06:46.319 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T07:06:46.319 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T07:06:46.320 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T07:06:46.321 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T07:06:46.320+0000 7f6f6ea63700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 49447) UID: 1000 2021-03-09T07:06:46.521 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:06:47.470 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:06:47.468+0000 7fc55f40e700 -1 rbd::mirror::ImageReplayer: 0x5614958fe780 [3/18a549b3-61f2-46e5-af4d-1a9b02c6e0ad] start: already running 2021-03-09T07:06:51.320 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T07:06:51.321 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:06:51.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:06:51.322 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T07:06:51.322 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T07:06:51.322 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T07:06:51.325 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T07:06:51.325 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T07:07:53.326 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4582654509398565) > weight (0.25) 2021-03-09T07:07:53.329 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T07:07:53.330 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T07:07:53.331 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T07:07:53.331 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T07:07:53.331 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:07:53.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T07:07:53.327+0000 7f53f2b54700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 52944) UID: 1000 2021-03-09T07:07:53.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:07:54.483 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.479+0000 7fc55f40e700 -1 rbd::mirror::ImageReplayer: 0x561491a13900 [3/08a00e86-016b-46cb-bca2-dcf2f528e189] start: already running 2021-03-09T07:07:54.483 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.479+0000 7fc55f40e700 -1 rbd::mirror::ImageReplayer: 0x561491a12c80 [3/0acd7cf2-0841-4351-804c-3096c0145f8f] start: already running 2021-03-09T07:07:54.483 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.479+0000 7fc55f40e700 -1 rbd::mirror::ImageReplayer: 0x5614958fef00 [3/18a549b3-61f2-46e5-af4d-1a9b02c6e0ad] start: already running 2021-03-09T07:07:54.485 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:07:54.479+0000 7f082a074700 -1 reset not still connected to 0x55a9f121d400 2021-03-09T07:07:54.487 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.483+0000 7fc55ec0d700 -1 rbd::mirror::ImageReplayer: 0x561491a12c80 [3/0acd7cf2-0841-4351-804c-3096c0145f8f] start: already running 2021-03-09T07:07:54.488 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.483+0000 7fc55f40e700 -1 rbd::mirror::ImageReplayer: 0x561491a13e00 [3/110cc5bb-34a2-4449-8efe-87f25e6218e2] start: already running 2021-03-09T07:07:54.488 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.487+0000 7fc55ec0d700 -1 rbd::mirror::ImageReplayer: 0x5614958fef00 [3/18a549b3-61f2-46e5-af4d-1a9b02c6e0ad] start: already running 2021-03-09T07:07:54.488 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.487+0000 7fc55f40e700 -1 rbd::mirror::ImageReplayer: 0x561491a13180 [3/28f924e2-e881-4421-bb68-c0bf69c20856] start: already running 2021-03-09T07:07:54.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.487+0000 7fc55f40e700 -1 rbd::mirror::ImageReplayer: 0x561491a14300 [3/6cd921d3-989d-483b-ba40-00f209ebbd60] start: already running 2021-03-09T07:07:54.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.487+0000 7fc55ec0d700 -1 rbd::mirror::ImageReplayer: 0x561491a13400 [3/7c5811a1-6f19-4292-9a3a-335262621df8] start: already running 2021-03-09T07:07:54.490 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.487+0000 7fc55ec0d700 -1 rbd::mirror::ImageReplayer: 0x561491a13b80 [3/7cda0e60-383d-4c8c-85dd-263c9aac913a] start: already running 2021-03-09T07:07:54.491 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.487+0000 7fc55f40e700 -1 rbd::mirror::ImageReplayer: 0x561491a13680 [3/a27c2afb-1b52-4788-afd8-b540c509149b] start: already running 2021-03-09T07:07:54.491 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.487+0000 7fc55ec0d700 -1 rbd::mirror::ImageReplayer: 0x561491a14580 [3/aec0a6af-f0ca-401c-a5d1-8f250674262a] start: already running 2021-03-09T07:07:54.491 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:07:54.487+0000 7fc55f40e700 -1 rbd::mirror::ImageReplayer: 0x561491a14080 [3/f250f852-7768-44dd-84f7-ce3818998e65] start: already running 2021-03-09T07:07:59.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:07:59.434 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T07:07:59.434 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T07:07:59.434 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T07:07:59.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T07:07:59.438 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T07:08:32.188 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:08:32.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:08:32.192 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T07:08:32.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:08:32.190+0000 7f153302e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 53422) UID: 1000 2021-03-09T07:08:32.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:08:39.193 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:08:39.193 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:08:39.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:08:39.194 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:08:39.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:08:39.195 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:08:39.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:08:39.201 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T07:08:39.434 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T07:08:39.428+0000 7fdda4a8d700 -1 reset not still connected to 0x55aefb276220 2021-03-09T07:08:39.434 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T07:08:39.428+0000 7fdda4a8d700 -1 reset not still connected to 0x55af004ba150 2021-03-09T07:08:39.434 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T07:08:39.428+0000 7fdda4a8d700 -1 reset not still connected to 0x55af004baf20 2021-03-09T07:08:39.435 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T07:08:39.428+0000 7fdda4a8d700 -1 reset not still connected to 0x55af01306150 2021-03-09T07:09:41.439 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7650206015782559) > weight (0.25) 2021-03-09T07:09:41.458 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3058014566463372) > weight (0.25) 2021-03-09T07:09:41.458 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7824825307965324) > weight (0.25) 2021-03-09T07:09:41.459 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T07:09:41.460 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T07:09:41.460 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T07:09:41.461 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T07:09:41.461+0000 7f675b582700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 53521) UID: 1000 2021-03-09T07:09:41.661 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:09:42.461 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T07:09:42.462 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:09:42.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:09:42.462 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T07:09:42.463 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T07:09:42.463 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T07:09:42.466 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T07:09:42.467 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T07:10:03.203 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:10:03.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:10:03.210 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T07:10:03.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:10:03.208+0000 7f608e845700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 53706) UID: 1000 2021-03-09T07:10:03.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:10:11.211 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:10:11.212 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:10:11.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:10:11.213 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:10:11.213 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:10:11.213 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:10:11.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:10:11.217 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T07:11:12.468 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2520596051553571) > weight (0.25) 2021-03-09T07:11:12.471 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T07:11:12.472 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T07:11:12.473 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T07:11:12.474 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T07:11:12.471+0000 7f795b117700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 53615) UID: 1000 2021-03-09T07:11:12.676 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:11:16.474 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T07:11:16.476 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:11:16.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:11:16.477 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T07:11:16.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T07:11:16.478 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T07:11:16.481 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T07:11:16.481 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T07:11:28.218 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7220462886451239) > weight (0.25) 2021-03-09T07:11:28.220 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T07:11:28.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T07:11:28.221 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T07:11:28.222 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T07:11:28.219+0000 7f9ae82b2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 52844) UID: 1000 2021-03-09T07:11:28.422 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:11:34.221 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T07:11:34.223 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:11:34.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:11:34.223 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T07:11:34.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T07:11:34.224 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T07:11:34.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T07:11:34.238 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-03-09T07:11:41.681 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:11:41.679+0000 7fc55f40e700 -1 rbd::mirror::ImageReplayer: 0x561495c2b400 [3/941e8ddf-64f0-4f25-89c8-9cfbbec2992f] start: already running 2021-03-09T07:11:41.682 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:11:41.679+0000 7fc55ec0d700 -1 rbd::mirror::ImageReplayer: 0x561495c2b680 [3/cbb866ff-7e4b-4de9-80d6-4f875228d852] start: already running 2021-03-09T07:11:41.694 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:11:41.695+0000 7f901a51c700 -1 reset not still connected to 0x55890173d670 2021-03-09T07:12:01.530 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:12:01.526+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf1fb0 2021-03-09T07:12:01.531 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:12:01.526+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf3dc0 2021-03-09T07:12:01.531 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:12:01.526+0000 7f901a51c700 -1 reset not still connected to 0x558904593e90 2021-03-09T07:12:49.482 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.56570979907481) > weight (0.25) 2021-03-09T07:12:49.506 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.39907910020961546) > weight (0.25) 2021-03-09T07:12:49.507 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4006272701126017) > weight (0.25) 2021-03-09T07:12:49.507 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3582358074844726) > weight (0.25) 2021-03-09T07:12:49.507 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T07:12:53.239 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5303739000018514) > weight (0.25) 2021-03-09T07:12:53.244 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T07:12:53.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T07:12:53.246 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T07:12:53.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T07:12:53.241+0000 7f516a6e4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 54088) UID: 1000 2021-03-09T07:12:53.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:13:01.246 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T07:13:01.248 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:13:01.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:13:01.249 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T07:13:01.249 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T07:13:01.250 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T07:13:01.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T07:13:01.253 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T07:14:20.508 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T07:14:20.512 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T07:14:20.513 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T07:14:20.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T07:14:20.512+0000 7f95b8493700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 53331) UID: 1000 2021-03-09T07:14:20.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:14:24.514 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T07:14:24.514 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:14:24.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:14:24.515 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T07:14:24.515 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T07:14:24.516 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T07:14:24.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T07:14:24.519 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T07:14:33.254 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7261480868244813) > weight (0.25) 2021-03-09T07:14:33.255 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T07:14:33.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T07:14:33.256 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T07:14:33.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T07:14:33.256+0000 7f642b9af700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 54190) UID: 1000 2021-03-09T07:14:33.457 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:14:41.257 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T07:14:41.259 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:14:41.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:14:41.260 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T07:14:41.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T07:14:41.262 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T07:14:41.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T07:14:41.265 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T07:15:36.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4490708992915221) > weight (0.25) 2021-03-09T07:15:36.524 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5700604103870858) > weight (0.25) 2021-03-09T07:15:36.524 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.25661720607785854) > weight (0.25) 2021-03-09T07:15:36.525 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9822873402472878) > weight (0.25) 2021-03-09T07:15:36.525 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T07:15:48.267 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6964512846611385) > weight (0.25) 2021-03-09T07:15:48.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8981357229265224) > weight (0.25) 2021-03-09T07:15:48.271 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T07:15:48.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T07:15:48.273 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T07:15:48.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T07:15:48.270+0000 7f2fafd80700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 53229) UID: 1000 2021-03-09T07:15:48.474 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:15:49.274 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T07:15:49.275 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:15:49.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:15:49.276 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T07:15:49.276 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T07:15:49.277 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T07:15:49.280 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T07:15:49.281 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T07:16:54.282 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:16:54.323 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:16:54.323 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T07:16:54.324 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:16:54.321+0000 7f1530ffb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 53898) UID: 1000 2021-03-09T07:16:54.525 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:17:02.324 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:17:02.325 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:17:02.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:17:02.326 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:17:02.326 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:17:02.326 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:17:02.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:17:02.329 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-03-09T07:17:08.526 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8520383377544263) > weight (0.25) 2021-03-09T07:17:08.528 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.831582448818539) > weight (0.25) 2021-03-09T07:17:08.529 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6206869347269107) > weight (0.25) 2021-03-09T07:17:08.529 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6631226142734481) > weight (0.25) 2021-03-09T07:17:08.529 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T07:18:23.331 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:18:23.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:18:23.336 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T07:18:23.336 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:18:23.336 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:18:23.337 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:18:23.332+0000 7f1ec602d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 54586) UID: 1000 2021-03-09T07:18:23.421 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:18:23.416+0000 7f082a074700 -1 reset not still connected to 0x55a99af5e270 2021-03-09T07:18:23.536 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:18:29.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:18:29.446 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:18:29.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:18:29.447 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:18:29.449 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:18:29.450 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-03-09T07:18:56.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.25760518762555695) > weight (0.25) 2021-03-09T07:18:56.546 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7120163531125346) > weight (0.25) 2021-03-09T07:18:56.546 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.818319543403261) > weight (0.25) 2021-03-09T07:18:56.547 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6760420012773382) > weight (0.25) 2021-03-09T07:18:56.547 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T07:19:37.451 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6077145673748335) > weight (0.25) 2021-03-09T07:19:37.454 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8470337019581653) > weight (0.25) 2021-03-09T07:19:37.454 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9581726640385488) > weight (0.25) 2021-03-09T07:19:37.454 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.49655595568024247) > weight (0.25) 2021-03-09T07:19:37.455 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T07:20:49.548 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T07:20:49.578 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T07:20:49.578 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T07:20:49.579 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T07:20:49.577+0000 7fd56e072700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 54289) UID: 1000 2021-03-09T07:20:49.780 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:20:53.579 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T07:20:53.580 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:20:53.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:20:53.581 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T07:20:53.581 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T07:20:53.581 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T07:20:53.584 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T07:20:53.585 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T07:21:06.456 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6135752212491177) > weight (0.25) 2021-03-09T07:21:06.459 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T07:21:06.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T07:21:06.460 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T07:21:06.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T07:21:06.457+0000 7fd329576700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 54381) UID: 1000 2021-03-09T07:21:06.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:21:08.461 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T07:21:08.462 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:21:08.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:21:08.463 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T07:21:08.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T07:21:08.464 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T07:21:08.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T07:21:08.468 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T07:21:35.961 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:21:35.958+0000 7f901a51c700 -1 reset not still connected to 0x558904590c30 2021-03-09T07:21:35.962 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:21:35.958+0000 7f901a51c700 -1 reset not still connected to 0x5589045912b0 2021-03-09T07:21:35.962 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:21:35.958+0000 7f901a51c700 -1 reset not still connected to 0x558904ab1c70 2021-03-09T07:22:16.723 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:22:16.720+0000 7f901a51c700 -1 reset not still connected to 0x5588f923dee0 2021-03-09T07:22:18.469 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5930191344821724) > weight (0.25) 2021-03-09T07:22:18.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.548826709500744) > weight (0.25) 2021-03-09T07:22:18.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.71428773898188) > weight (0.25) 2021-03-09T07:22:18.471 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6534351360718554) > weight (0.25) 2021-03-09T07:22:18.471 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T07:22:39.585 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T07:22:39.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T07:22:39.609 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T07:22:39.610 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T07:22:39.607+0000 7f0fb5183700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 54804) UID: 1000 2021-03-09T07:22:39.810 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:22:42.610 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T07:22:42.611 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:22:42.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:22:42.612 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T07:22:42.612 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T07:22:42.612 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T07:22:42.615 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T07:22:42.615 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T07:23:35.472 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:23:35.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:23:35.481 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T07:23:35.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:23:35.479+0000 7f82928b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 54686) UID: 1000 2021-03-09T07:23:35.681 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:23:36.482 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:23:36.482 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:23:36.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:23:36.483 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:23:36.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:23:36.484 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:23:36.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:23:36.487 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T07:23:58.616 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.30965590886671157) > weight (0.25) 2021-03-09T07:23:58.619 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4901582479555542) > weight (0.25) 2021-03-09T07:23:58.619 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3378110077767439) > weight (0.25) 2021-03-09T07:23:58.620 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7726406775932666) > weight (0.25) 2021-03-09T07:23:58.620 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T07:24:49.488 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.998028881876396) > weight (0.25) 2021-03-09T07:24:49.518 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T07:24:49.519 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T07:24:49.519 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T07:24:49.520 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T07:24:49.517+0000 7f4ff5cb9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 54895) UID: 1000 2021-03-09T07:24:49.721 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:24:51.521 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T07:24:51.530 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:24:51.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:24:51.530 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T07:24:51.531 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T07:24:51.531 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T07:24:51.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T07:24:51.535 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T07:25:01.621 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T07:25:01.625 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T07:25:01.628 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T07:25:01.629 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T07:25:01.629 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:25:01.644 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T07:25:01.629+0000 7f5019a91700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 54994) UID: 1000 2021-03-09T07:25:01.836 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:25:07.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:25:07.743 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T07:25:07.744 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T07:25:07.744 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T07:25:07.747 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T07:25:07.747 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T07:26:23.748 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7481251713146502) > weight (0.25) 2021-03-09T07:26:23.752 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4846729218122018) > weight (0.25) 2021-03-09T07:26:23.752 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.624855248910409) > weight (0.25) 2021-03-09T07:26:23.752 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6516831387294211) > weight (0.25) 2021-03-09T07:26:23.752 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T07:26:30.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7476675397958029) > weight (0.25) 2021-03-09T07:26:30.539 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7156803861619139) > weight (0.25) 2021-03-09T07:26:30.539 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4607382657653646) > weight (0.25) 2021-03-09T07:26:30.540 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5950340785665275) > weight (0.25) 2021-03-09T07:26:30.540 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T07:26:42.690 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:26:42.687+0000 7f082a074700 -1 reset not still connected to 0x55a98f9cf260 2021-03-09T07:26:42.691 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:26:42.687+0000 7f082a074700 -1 reset not still connected to 0x55a996c45330 2021-03-09T07:26:42.691 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:26:42.687+0000 7f082a074700 -1 reset not still connected to 0x55a9f121b520 2021-03-09T07:27:11.327 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T07:27:11.324+0000 7fdda4a8d700 -1 reset not still connected to 0x55af004b8a90 2021-03-09T07:27:11.328 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T07:27:11.324+0000 7fdda4a8d700 -1 reset not still connected to 0x55af004b9930 2021-03-09T07:27:47.541 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:27:47.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:27:47.550 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T07:27:47.551 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:27:47.550+0000 7f6a91d08700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 55085) UID: 1000 2021-03-09T07:27:47.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:27:48.550 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:27:48.551 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:27:48.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:27:48.552 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:27:48.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:27:48.552 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:27:48.555 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:27:48.555 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T07:27:57.753 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42554043579852174) > weight (0.25) 2021-03-09T07:27:57.755 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8027063339426845) > weight (0.25) 2021-03-09T07:27:57.755 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T07:27:57.756 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T07:27:57.757 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T07:27:57.757 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T07:27:57.757 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:27:57.758 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:27:57.754+0000 7fc55fc0f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 53037) UID: 1000 2021-03-09T07:27:57.957 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:28:03.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:28:03.859 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T07:28:03.860 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T07:28:03.860 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T07:28:03.873 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T07:28:03.873 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T07:28:57.557 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6614194812517724) > weight (0.25) 2021-03-09T07:28:57.560 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8782605909101654) > weight (0.25) 2021-03-09T07:28:57.560 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.46901896259455056) > weight (0.25) 2021-03-09T07:28:57.560 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7496093936661835) > weight (0.25) 2021-03-09T07:28:57.560 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T07:29:57.874 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T07:29:57.877 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T07:29:57.877 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T07:29:57.878 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T07:29:57.878 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:29:57.879 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T07:29:57.876+0000 7fb14deb6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 55287) UID: 1000 2021-03-09T07:29:58.079 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:29:59.562 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3397759469486432) > weight (0.25) 2021-03-09T07:29:59.562 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6303839420767863) > weight (0.25) 2021-03-09T07:29:59.562 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T07:29:59.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T07:29:59.563 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T07:29:59.564 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T07:29:59.564+0000 7fc896f92700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 54484) UID: 1000 2021-03-09T07:29:59.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:30:03.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:30:03.980 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T07:30:03.980 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T07:30:03.981 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T07:30:03.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T07:30:03.983 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T07:30:04.564 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T07:30:04.564 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:30:04.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:30:04.565 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T07:30:04.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T07:30:04.566 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T07:30:04.568 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T07:30:04.568 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T07:30:16.711 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:30:16.708+0000 7f082a074700 -1 reset not still connected to 0x55a9cf585930 2021-03-09T07:30:16.713 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:30:16.712+0000 7f082a074700 -1 reset not still connected to 0x55a9cf586220 2021-03-09T07:30:16.714 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:30:16.712+0000 7f082a074700 -1 reset not still connected to 0x55a9f04ea8a0 2021-03-09T07:31:05.569 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4057866443001855) > weight (0.25) 2021-03-09T07:31:05.571 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3499638197646229) > weight (0.25) 2021-03-09T07:31:05.571 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T07:31:05.572 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T07:31:05.572 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T07:31:05.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T07:31:05.571+0000 7fceac775700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 55662) UID: 1000 2021-03-09T07:31:05.773 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:31:06.573 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T07:31:06.574 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:31:06.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:31:06.574 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T07:31:06.575 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T07:31:06.575 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T07:31:06.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T07:31:06.578 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T07:31:58.985 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T07:31:58.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T07:31:58.988 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T07:31:58.989 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T07:31:58.986+0000 7f3d0465b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 55571) UID: 1000 2021-03-09T07:31:59.189 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:32:00.989 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T07:32:00.995 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:32:00.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:32:00.996 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T07:32:00.996 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T07:32:00.997 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T07:32:00.999 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T07:32:01.000 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T07:33:03.581 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32628996040701763) > weight (0.25) 2021-03-09T07:33:03.626 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.36193862349830697) > weight (0.25) 2021-03-09T07:33:03.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9723975320925494) > weight (0.25) 2021-03-09T07:33:03.627 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9467005872390948) > weight (0.25) 2021-03-09T07:33:03.627 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T07:33:04.001 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7214727346849279) > weight (0.25) 2021-03-09T07:33:04.001 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.38010421467678823) > weight (0.25) 2021-03-09T07:33:04.001 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2790534040761744) > weight (0.25) 2021-03-09T07:33:04.002 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8939863285433938) > weight (0.25) 2021-03-09T07:33:04.002 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T07:34:21.003 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8287873273031078) > weight (0.25) 2021-03-09T07:34:21.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8203065258833006) > weight (0.25) 2021-03-09T07:34:21.007 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9301453662250165) > weight (0.25) 2021-03-09T07:34:21.007 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6298050809340879) > weight (0.25) 2021-03-09T07:34:21.007 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T07:34:48.629 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:34:48.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:34:48.649 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T07:34:48.650 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:34:48.647+0000 7f278766a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 55381) UID: 1000 2021-03-09T07:34:48.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:34:56.651 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:34:56.651 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:34:56.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:34:56.652 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:34:56.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:34:56.653 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:34:56.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:34:56.656 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-03-09T07:35:16.531 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:35:16.530+0000 7f901a51c700 -1 reset not still connected to 0x5588fa39b670 2021-03-09T07:35:16.532 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:35:16.530+0000 7f901a51c700 -1 reset not still connected to 0x5588fb944d00 2021-03-09T07:35:16.532 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:35:16.530+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf15f0 2021-03-09T07:35:16.532 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:35:16.530+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf1860 2021-03-09T07:35:16.533 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:35:16.530+0000 7f901a51c700 -1 reset not still connected to 0x558903e2a8f0 2021-03-09T07:35:16.533 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:35:16.530+0000 7f901a51c700 -1 reset not still connected to 0x558903e2bfb0 2021-03-09T07:35:16.534 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:35:16.530+0000 7f901a51c700 -1 reset not still connected to 0x558903e2db50 2021-03-09T07:36:11.008 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T07:36:11.012 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T07:36:11.013 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T07:36:11.013 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T07:36:11.010+0000 7f87f9895700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 55864) UID: 1000 2021-03-09T07:36:11.213 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:36:14.657 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9750423088454983) > weight (0.25) 2021-03-09T07:36:14.661 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8221954319182261) > weight (0.25) 2021-03-09T07:36:14.661 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3893011099415812) > weight (0.25) 2021-03-09T07:36:14.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6615798965082413) > weight (0.25) 2021-03-09T07:36:14.662 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-03-09T07:36:17.014 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T07:36:17.014 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:36:17.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:36:17.015 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T07:36:17.015 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T07:36:17.016 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T07:36:17.019 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T07:36:17.019 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T07:37:45.020 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9008261885767658) > weight (0.25) 2021-03-09T07:37:45.052 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7463612926564004) > weight (0.25) 2021-03-09T07:37:45.053 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4761358178704912) > weight (0.25) 2021-03-09T07:37:45.053 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T07:37:45.054 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T07:37:45.054 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T07:37:45.055 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T07:37:45.052+0000 7fda47855700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 53806) UID: 1000 2021-03-09T07:37:45.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:37:50.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.34831704757901005) > weight (0.25) 2021-03-09T07:37:50.664 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45970807076999753) > weight (0.25) 2021-03-09T07:37:50.664 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T07:37:50.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T07:37:50.665 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T07:37:50.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T07:37:50.664+0000 7fadb3fb7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 55765) UID: 1000 2021-03-09T07:37:50.866 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:37:54.056 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T07:37:54.056 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:37:54.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:37:54.057 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T07:37:54.057 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T07:37:54.058 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T07:37:54.060 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T07:37:54.061 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-03-09T07:37:55.666 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T07:37:55.666 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:37:55.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:37:55.667 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T07:37:55.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T07:37:55.667 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T07:37:55.672 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T07:37:55.672 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T07:39:09.674 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:39:09.745 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:39:09.752 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T07:39:09.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:39:09.742+0000 7f00c041c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 55957) UID: 1000 2021-03-09T07:39:09.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:39:13.753 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:39:13.753 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:39:13.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:39:13.754 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:39:13.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:39:13.755 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:39:13.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:39:13.758 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T07:39:29.062 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T07:39:29.064 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T07:39:29.064 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T07:39:29.065 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T07:39:29.062+0000 7fac6fdcc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 56061) UID: 1000 2021-03-09T07:39:29.266 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:39:37.066 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T07:39:37.066 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:39:37.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:39:37.067 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T07:39:37.067 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T07:39:37.068 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T07:39:37.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T07:39:37.071 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T07:40:26.759 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.25092949104197426) > weight (0.25) 2021-03-09T07:40:26.762 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2505231292551353) > weight (0.25) 2021-03-09T07:40:26.763 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.416623387872355) > weight (0.25) 2021-03-09T07:40:26.763 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.402882956655209) > weight (0.25) 2021-03-09T07:40:26.763 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T07:41:16.072 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8480709170417424) > weight (0.25) 2021-03-09T07:41:16.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7594272334674039) > weight (0.25) 2021-03-09T07:41:16.102 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T07:41:16.103 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T07:41:16.104 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T07:41:16.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:41:16.104+0000 7f0674ae5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 55480) UID: 1000 2021-03-09T07:41:16.305 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:41:21.105 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T07:41:21.115 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:41:21.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:41:21.116 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T07:41:21.117 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T07:41:21.120 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T07:41:21.142 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T07:41:21.143 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T07:41:24.305 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:41:24.300+0000 7f082a074700 -1 reset not still connected to 0x55a9f04eb670 2021-03-09T07:42:18.764 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:42:18.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:42:18.770 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T07:42:18.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:42:18.767+0000 7fb5f970b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 56343) UID: 1000 2021-03-09T07:42:18.971 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:42:20.772 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:42:20.773 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:42:20.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:42:20.774 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:42:20.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:42:20.776 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:42:20.781 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:42:20.781 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T07:42:24.146 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.461352671154441) > weight (0.25) 2021-03-09T07:42:24.147 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6883405595073088) > weight (0.25) 2021-03-09T07:42:24.148 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T07:42:24.148 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T07:42:24.148 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T07:42:24.152 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:42:24.151+0000 7efcdb585700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 56535) UID: 1000 2021-03-09T07:42:24.353 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:42:26.149 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T07:42:26.150 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:42:26.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:42:26.151 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T07:42:26.151 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T07:42:26.152 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T07:42:26.154 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T07:42:26.155 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T07:42:56.623 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:42:56.587+0000 7f082a074700 -1 reset not still connected to 0x55a9cf585930 2021-03-09T07:42:56.624 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:42:56.587+0000 7f082a074700 -1 reset not still connected to 0x55a9cf586220 2021-03-09T07:42:56.624 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:42:56.587+0000 7f082a074700 -1 reset not still connected to 0x55a9f04ea8a0 2021-03-09T07:43:27.783 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7404956184029607) > weight (0.25) 2021-03-09T07:43:27.791 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.37492586713843845) > weight (0.25) 2021-03-09T07:43:27.791 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T07:43:27.792 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T07:43:27.793 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T07:43:27.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T07:43:27.790+0000 7f74993e5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 56244) UID: 1000 2021-03-09T07:43:27.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:43:34.794 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T07:43:34.796 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:43:34.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:43:34.797 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T07:43:34.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T07:43:34.798 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T07:43:34.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T07:43:34.801 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T07:44:06.157 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6358868501033751) > weight (0.25) 2021-03-09T07:44:06.159 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8590653744625195) > weight (0.25) 2021-03-09T07:44:06.159 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6309640462176914) > weight (0.25) 2021-03-09T07:44:06.159 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T07:44:06.160 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T07:44:06.161 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T07:44:06.161 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T07:44:06.161 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:44:06.162 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T07:44:06.157+0000 7f5fd1406700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 56153) UID: 1000 2021-03-09T07:44:06.170 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T07:44:06.165+0000 7fc0e0d45700 -1 reset not still connected to 0x55e8fb25ba80 2021-03-09T07:44:06.362 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:44:12.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:44:12.263 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T07:44:12.264 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T07:44:12.264 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T07:44:12.268 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T07:44:12.268 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T07:45:27.802 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.761313130198004) > weight (0.25) 2021-03-09T07:45:27.827 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T07:45:27.828 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T07:45:27.828 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T07:45:27.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T07:45:27.828+0000 7f1236e7f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 55184) UID: 1000 2021-03-09T07:45:28.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:45:36.829 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T07:45:36.832 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:45:36.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:45:36.832 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T07:45:36.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T07:45:36.833 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T07:45:36.836 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T07:45:36.836 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T07:45:41.269 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.48325344948717264) > weight (0.25) 2021-03-09T07:45:41.269 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4558474264688811) > weight (0.25) 2021-03-09T07:45:41.270 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T07:45:41.270 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T07:45:41.271 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T07:45:41.272 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:45:41.268+0000 7f4ad895a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 56718) UID: 1000 2021-03-09T07:45:41.472 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:45:48.272 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T07:45:48.273 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:45:48.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:45:48.274 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T07:45:48.274 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T07:45:48.275 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T07:45:48.279 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T07:45:48.279 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T07:45:57.918 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:45:57.911+0000 7f082a074700 -1 reset not still connected to 0x55a990b878e0 2021-03-09T07:45:57.919 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:45:57.915+0000 7f082a074700 -1 reset not still connected to 0x55a99f963110 2021-03-09T07:46:56.837 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:46:56.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:46:56.846 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T07:46:56.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:46:56.842+0000 7fcc2d444700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 56627) UID: 1000 2021-03-09T07:46:57.048 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:47:00.848 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:47:00.849 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:47:00.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:47:00.851 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:47:00.851 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:47:00.856 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:47:00.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:47:00.860 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T07:47:21.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8771757952016458) > weight (0.25) 2021-03-09T07:47:21.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.644057785385853) > weight (0.25) 2021-03-09T07:47:21.282 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T07:47:21.283 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T07:47:21.284 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T07:47:21.285 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T07:47:21.282+0000 7f121ec94700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 57180) UID: 1000 2021-03-09T07:47:21.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:47:26.285 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T07:47:26.419 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:47:26.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:47:26.420 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T07:47:26.421 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T07:47:26.421 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T07:47:26.444 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T07:47:26.445 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T07:48:55.861 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8293847175443123) > weight (0.25) 2021-03-09T07:48:55.888 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3567422919008525) > weight (0.25) 2021-03-09T07:48:55.888 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4636287587859905) > weight (0.25) 2021-03-09T07:48:55.889 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5179511924484196) > weight (0.25) 2021-03-09T07:48:55.889 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T07:49:08.446 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6649804478330079) > weight (0.25) 2021-03-09T07:49:08.447 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.494233976689727) > weight (0.25) 2021-03-09T07:49:08.448 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9963958983468268) > weight (0.25) 2021-03-09T07:49:08.448 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T07:49:08.449 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T07:49:08.449 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T07:49:08.456 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T07:49:08.452+0000 7fd572ea2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 56985) UID: 1000 2021-03-09T07:49:08.656 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:49:16.450 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T07:49:16.451 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:49:16.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:49:16.452 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T07:49:16.452 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T07:49:16.452 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T07:49:16.455 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T07:49:16.455 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T07:49:35.865 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:49:35.860+0000 7f082a074700 -1 reset not still connected to 0x55a9cf585930 2021-03-09T07:49:35.865 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T07:49:35.860+0000 7f2c472a6700 -1 rbd::mirror::ImageReplayer: 0x55da14098f80 [3/8853c04a-4df4-47fb-9df9-6cebf19cc680] start: already running 2021-03-09T07:49:35.866 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T07:49:35.860+0000 7f2c472a6700 -1 rbd::mirror::ImageReplayer: 0x55da14097b80 [3/bd4ef0c3-a3a7-42f2-8034-c828cab83717] start: already running 2021-03-09T07:49:35.870 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:49:35.868+0000 7f082a074700 -1 reset not still connected to 0x55a9cf586220 2021-03-09T07:49:35.870 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:49:35.868+0000 7f082a074700 -1 reset not still connected to 0x55a9f04ea8a0 2021-03-09T07:50:15.890 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3686161563424384) > weight (0.25) 2021-03-09T07:50:15.892 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6421605166480162) > weight (0.25) 2021-03-09T07:50:15.893 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8381025901214102) > weight (0.25) 2021-03-09T07:50:15.893 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7678188914155203) > weight (0.25) 2021-03-09T07:50:15.895 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T07:50:58.456 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43508083211969506) > weight (0.25) 2021-03-09T07:50:58.484 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4869313516257333) > weight (0.25) 2021-03-09T07:50:58.484 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.27227373996696247) > weight (0.25) 2021-03-09T07:50:58.484 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7408618109206387) > weight (0.25) 2021-03-09T07:50:58.485 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T07:51:38.896 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:51:38.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:51:38.898 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T07:51:38.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:51:38.897+0000 7fe194fef700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 57273) UID: 1000 2021-03-09T07:51:39.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:51:45.899 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:51:45.902 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:51:45.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:51:45.903 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:51:45.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:51:45.903 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:51:45.907 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:51:45.907 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-03-09T07:51:53.234 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:51:53.220+0000 7f901a51c700 -1 reset not still connected to 0x5588fb947740 2021-03-09T07:51:53.235 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:51:53.220+0000 7f901a51c700 -1 reset not still connected to 0x558903e2a270 2021-03-09T07:51:53.235 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:51:53.220+0000 7f901a51c700 -1 reset not still connected to 0x558903e2a750 2021-03-09T07:51:53.235 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:51:53.220+0000 7f901a51c700 -1 reset not still connected to 0x558903e2d670 2021-03-09T07:52:29.926 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:52:29.923+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf15f0 2021-03-09T07:52:30.452 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:52:29.927+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf2220 2021-03-09T07:52:30.452 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:52:29.931+0000 7f901a51c700 -1 reset not still connected to 0x558903e2db50 2021-03-09T07:52:44.485 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5358427064206033) > weight (0.25) 2021-03-09T07:52:44.486 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T07:52:44.487 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T07:52:44.487 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T07:52:44.488 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T07:52:44.484+0000 7f2c47aa7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 53997) UID: 1000 2021-03-09T07:52:44.688 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:52:46.488 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T07:52:46.490 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:52:46.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:52:46.491 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T07:52:46.491 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T07:52:46.492 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T07:52:46.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T07:52:46.495 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T07:53:20.908 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T07:53:20.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T07:53:20.909 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T07:53:20.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T07:53:20.908+0000 7fd8770bc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 57557) UID: 1000 2021-03-09T07:53:21.110 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:53:26.910 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T07:53:26.911 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:53:26.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:53:26.912 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T07:53:26.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T07:53:26.914 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T07:53:26.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T07:53:26.917 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T07:53:46.708 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:53:46.701+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf2be0 2021-03-09T07:54:18.496 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.44186481686977375) > weight (0.25) 2021-03-09T07:54:18.497 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.496889263450498) > weight (0.25) 2021-03-09T07:54:18.497 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.752074235950494) > weight (0.25) 2021-03-09T07:54:18.498 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3540135837995255) > weight (0.25) 2021-03-09T07:54:18.498 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T07:54:46.690 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:54:46.687+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf2be0 2021-03-09T07:55:05.918 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9492452772737037) > weight (0.25) 2021-03-09T07:55:05.918 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T07:55:05.919 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T07:55:05.920 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T07:55:05.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T07:55:05.914+0000 7f68b8dff700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 57081) UID: 1000 2021-03-09T07:55:06.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:55:10.920 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T07:55:10.930 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:55:10.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:55:10.931 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T07:55:10.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T07:55:10.931 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T07:55:10.938 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T07:55:10.939 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T07:55:16.283 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:16.278+0000 7f901a51c700 -1 reset not still connected to 0x5588f25515a0 2021-03-09T07:55:16.283 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:16.278+0000 7f901a51c700 -1 reset not still connected to 0x5588fa39b670 2021-03-09T07:55:16.283 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:16.278+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf15f0 2021-03-09T07:55:16.289 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:16.286+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf2220 2021-03-09T07:55:16.289 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:16.286+0000 7f901a51c700 -1 reset not still connected to 0x558903e2ce50 2021-03-09T07:55:16.289 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:16.286+0000 7f901a51c700 -1 reset not still connected to 0x558903e2db50 2021-03-09T07:55:37.380 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:55:37.373+0000 7f082a074700 -1 reset not still connected to 0x55a990b878e0 2021-03-09T07:55:37.381 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:55:37.373+0000 7f082a074700 -1 reset not still connected to 0x55a99af610c0 2021-03-09T07:55:37.381 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:55:37.373+0000 7f082a074700 -1 reset not still connected to 0x55a99db969c0 2021-03-09T07:55:37.381 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:55:37.373+0000 7f082a074700 -1 reset not still connected to 0x55a9f04eb670 2021-03-09T07:55:37.381 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T07:55:37.373+0000 7f082a074700 -1 reset not still connected to 0x55a9f121a820 2021-03-09T07:55:40.429 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:40.421+0000 7f901a51c700 -1 reset not still connected to 0x5588f25515a0 2021-03-09T07:55:40.429 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:40.421+0000 7f901a51c700 -1 reset not still connected to 0x5588f8cb3810 2021-03-09T07:55:40.429 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:40.421+0000 7f901a51c700 -1 reset not still connected to 0x5588f96f30c0 2021-03-09T07:55:40.430 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:40.421+0000 7f901a51c700 -1 reset not still connected to 0x5588fa39b670 2021-03-09T07:55:40.430 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:40.421+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf15f0 2021-03-09T07:55:40.430 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:40.425+0000 7f901a51c700 -1 reset not still connected to 0x558903e2ce50 2021-03-09T07:55:40.431 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T07:55:40.425+0000 7f901a51c700 -1 reset not still connected to 0x558903e2db50 2021-03-09T07:56:04.499 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6833461282743724) > weight (0.25) 2021-03-09T07:56:04.500 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T07:56:04.501 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T07:56:04.502 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T07:56:04.502 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T07:56:04.503 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:56:04.504 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T07:56:04.501+0000 7f931a08d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 57657) UID: 1000 2021-03-09T07:56:04.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:56:10.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:56:10.732 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T07:56:10.732 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T07:56:10.733 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T07:56:10.748 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T07:56:10.748 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-03-09T07:57:03.940 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9610961984197951) > weight (0.25) 2021-03-09T07:57:03.942 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5613173972599648) > weight (0.25) 2021-03-09T07:57:03.942 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T07:57:03.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T07:57:03.944 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T07:57:03.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T07:57:03.940+0000 7fcd2bcb8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 56819) UID: 1000 2021-03-09T07:57:04.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:57:11.945 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T07:57:11.946 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:57:11.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:57:11.947 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T07:57:11.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T07:57:11.948 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T07:57:11.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T07:57:11.951 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T07:57:48.749 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6011838197047537) > weight (0.25) 2021-03-09T07:57:48.751 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T07:57:48.752 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T07:57:48.752 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T07:57:48.753 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T07:57:48.747+0000 7f4dc1cf1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 57951) UID: 1000 2021-03-09T07:57:48.953 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:57:54.753 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T07:57:54.753 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:57:54.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:57:54.754 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T07:57:54.754 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T07:57:54.755 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T07:57:54.757 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T07:57:54.758 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T07:58:33.952 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6968843624697) > weight (0.25) 2021-03-09T07:58:33.954 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.755920742248049) > weight (0.25) 2021-03-09T07:58:33.954 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9003999759688895) > weight (0.25) 2021-03-09T07:58:33.955 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6247146840008121) > weight (0.25) 2021-03-09T07:58:33.955 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T07:59:18.759 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6078098199401216) > weight (0.25) 2021-03-09T07:59:18.806 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T07:59:18.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T07:59:18.807 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T07:59:18.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T07:59:18.806+0000 7f783b9d6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 58142) UID: 1000 2021-03-09T07:59:19.009 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T07:59:24.809 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T07:59:24.811 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T07:59:24.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T07:59:24.812 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T07:59:24.813 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T07:59:24.813 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T07:59:24.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T07:59:24.817 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T08:00:27.956 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T08:00:27.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T08:00:27.959 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T08:00:27.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:00:27.960+0000 7fcf549d5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 57749) UID: 1000 2021-03-09T08:00:28.163 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:00:32.960 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T08:00:32.961 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:00:32.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:00:32.961 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T08:00:32.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T08:00:32.962 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T08:00:32.965 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T08:00:32.965 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T08:01:01.817 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.36570807455639565) > weight (0.25) 2021-03-09T08:01:01.818 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5725318734649366) > weight (0.25) 2021-03-09T08:01:01.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7980129866571742) > weight (0.25) 2021-03-09T08:01:01.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7575268432527694) > weight (0.25) 2021-03-09T08:01:01.819 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T08:01:38.967 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8423204244484999) > weight (0.25) 2021-03-09T08:01:38.968 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6541719002408938) > weight (0.25) 2021-03-09T08:01:38.969 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T08:01:38.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T08:01:38.971 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T08:01:38.971 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T08:01:38.971 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:01:38.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T08:01:38.967+0000 7f59120e8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 58043) UID: 1000 2021-03-09T08:01:39.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:01:45.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:01:45.074 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T08:01:45.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T08:01:45.075 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T08:01:45.078 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T08:01:45.078 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T08:02:06.820 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:02:06.849 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:02:06.850 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T08:02:06.851 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:02:06.847+0000 7fefb2f94700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 56442) UID: 1000 2021-03-09T08:02:07.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:02:07.850 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:02:07.851 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:02:07.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:02:07.863 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:02:07.866 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:02:07.873 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:02:07.908 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:02:07.909 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T08:03:22.079 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T08:03:22.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T08:03:22.083 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T08:03:22.084 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T08:03:22.084 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:03:22.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:03:22.077+0000 7f6d12613700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58326) UID: 1000 2021-03-09T08:03:22.288 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:03:28.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:03:28.189 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T08:03:28.189 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T08:03:28.190 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T08:03:28.193 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T08:03:28.194 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T08:03:55.911 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:03:55.940 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:03:55.941 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T08:03:55.941 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:03:55.941 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:03:55.943 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:03:55.933+0000 7f5563a4a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 58526) UID: 1000 2021-03-09T08:03:56.141 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:04:02.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:04:02.047 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:04:02.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:04:02.048 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:04:02.051 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:04:02.052 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T08:05:10.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.318323311254483) > weight (0.25) 2021-03-09T08:05:10.199 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T08:05:10.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T08:05:10.201 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T08:05:10.201 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T08:05:10.195+0000 7ff43127c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 57851) UID: 1000 2021-03-09T08:05:10.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:05:15.202 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T08:05:15.203 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:05:15.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:05:15.204 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T08:05:15.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T08:05:15.206 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T08:05:15.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T08:05:15.213 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T08:05:52.053 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6444043600285746) > weight (0.25) 2021-03-09T08:05:52.083 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T08:05:52.084 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T08:05:52.084 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T08:05:52.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T08:05:52.083+0000 7f0c05cde700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 58234) UID: 1000 2021-03-09T08:05:52.285 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:05:54.085 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T08:05:54.085 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:05:54.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:05:54.086 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T08:05:54.086 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T08:05:54.087 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T08:05:54.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T08:05:54.105 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T08:06:27.214 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.42541232606021195) > weight (0.25) 2021-03-09T08:06:27.216 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T08:06:27.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T08:06:27.217 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T08:06:27.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T08:06:27.214+0000 7f3376cd3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 58812) UID: 1000 2021-03-09T08:06:27.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:06:32.219 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T08:06:32.219 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:06:32.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:06:32.220 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T08:06:32.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T08:06:32.221 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T08:06:32.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T08:06:32.224 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-03-09T08:07:45.107 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33230219039629416) > weight (0.25) 2021-03-09T08:07:45.107 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7735997126324914) > weight (0.25) 2021-03-09T08:07:45.108 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7146474813980191) > weight (0.25) 2021-03-09T08:07:45.108 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7445558911732786) > weight (0.25) 2021-03-09T08:07:45.109 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T08:08:12.464 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:08:12.460+0000 7f082a074700 -1 reset not still connected to 0x55a98e817a00 2021-03-09T08:08:12.490 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:08:12.464+0000 7f082a074700 -1 reset not still connected to 0x55a990e9d400 2021-03-09T08:08:12.490 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:08:12.468+0000 7f082a074700 -1 reset not still connected to 0x55a9f121ad00 2021-03-09T08:08:30.226 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.46593742860090126) > weight (0.25) 2021-03-09T08:08:30.230 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9704098576136637) > weight (0.25) 2021-03-09T08:08:30.230 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T08:08:30.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T08:08:30.232 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T08:08:30.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T08:08:30.228+0000 7f67a2275700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 58427) UID: 1000 2021-03-09T08:08:30.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:08:32.233 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T08:08:32.234 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:08:32.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:08:32.235 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T08:08:32.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T08:08:32.236 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T08:08:32.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T08:08:32.240 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T08:08:57.949 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:08:57.946+0000 7f901a51c700 -1 reset not still connected to 0x558903e2db50 2021-03-09T08:09:14.109 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4401173020410102) > weight (0.25) 2021-03-09T08:09:14.111 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T08:09:14.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T08:09:14.112 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T08:09:14.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T08:09:14.111+0000 7f4d57a52700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 58911) UID: 1000 2021-03-09T08:09:14.313 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:09:15.113 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T08:09:15.113 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:09:15.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:09:15.114 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T08:09:15.114 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T08:09:15.115 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T08:09:15.118 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T08:09:15.118 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T08:10:21.241 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5801263016096818) > weight (0.25) 2021-03-09T08:10:21.243 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T08:10:21.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T08:10:21.245 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T08:10:21.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T08:10:21.246+0000 7f4da6f0b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 59002) UID: 1000 2021-03-09T08:10:21.451 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:10:30.245 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T08:10:30.246 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:10:30.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:10:30.246 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T08:10:30.247 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T08:10:30.247 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T08:10:30.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T08:10:30.250 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-03-09T08:10:32.487 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:10:32.482+0000 7f082a074700 -1 reset not still connected to 0x55a9935b5ad0 2021-03-09T08:10:32.492 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:10:32.486+0000 7f082a074700 -1 reset not still connected to 0x55a9935b7c20 2021-03-09T08:10:32.493 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:10:32.486+0000 7f082a074700 -1 reset not still connected to 0x55aa12b215a0 2021-03-09T08:11:07.118 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:11:07.167 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:11:07.168 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T08:11:07.169 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:11:07.165+0000 7f893baa4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 58718) UID: 1000 2021-03-09T08:11:07.369 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:11:09.169 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:11:09.170 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:11:09.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:11:09.170 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:11:09.171 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:11:09.171 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:11:09.174 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:11:09.174 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T08:11:25.827 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:11:25.822+0000 7f901a51c700 -1 reset not still connected to 0x5588f8a22490 2021-03-09T08:11:25.828 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:11:25.822+0000 7f901a51c700 -1 reset not still connected to 0x5588f923fc20 2021-03-09T08:11:25.828 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:11:25.822+0000 7f901a51c700 -1 reset not still connected to 0x5588fa39b670 2021-03-09T08:11:25.828 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:11:25.822+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf15f0 2021-03-09T08:11:25.832 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:11:25.826+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf2220 2021-03-09T08:11:25.832 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:11:25.826+0000 7f901a51c700 -1 reset not still connected to 0x558903e2db50 2021-03-09T08:12:08.251 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.37061030115273663) > weight (0.25) 2021-03-09T08:12:08.253 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7408312065353553) > weight (0.25) 2021-03-09T08:12:08.253 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4398039716302944) > weight (0.25) 2021-03-09T08:12:08.254 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4056404632230175) > weight (0.25) 2021-03-09T08:12:08.254 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T08:12:13.234 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T08:12:13.228+0000 7fdda4a8d700 -1 reset not still connected to 0x55af03a150c0 2021-03-09T08:13:05.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9946306404495858) > weight (0.25) 2021-03-09T08:13:05.210 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T08:13:05.211 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T08:13:05.211 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T08:13:05.212 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T08:13:05.212 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:13:05.213 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T08:13:05.207+0000 7f2c5a050700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 59204) UID: 1000 2021-03-09T08:13:05.412 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:13:11.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:13:11.314 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T08:13:11.314 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T08:13:11.315 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T08:13:11.317 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T08:13:11.318 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T08:13:37.255 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2919625918689851) > weight (0.25) 2021-03-09T08:13:37.258 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6127424053495006) > weight (0.25) 2021-03-09T08:13:37.258 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T08:13:37.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T08:13:37.259 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T08:13:37.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T08:13:37.255+0000 7efef15b2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 59104) UID: 1000 2021-03-09T08:13:37.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:13:40.260 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T08:13:40.268 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:13:40.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:13:40.270 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T08:13:40.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T08:13:40.271 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T08:13:40.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T08:13:40.275 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T08:14:21.319 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35236832573304133) > weight (0.25) 2021-03-09T08:14:21.320 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8712118787902388) > weight (0.25) 2021-03-09T08:14:21.320 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T08:14:21.321 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T08:14:21.321 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T08:14:21.322 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T08:14:21.318+0000 7f87564e3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 57372) UID: 1000 2021-03-09T08:14:21.523 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:14:28.323 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T08:14:28.323 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:14:28.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:14:28.325 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T08:14:28.325 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T08:14:28.326 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T08:14:28.330 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T08:14:28.331 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T08:14:54.275 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7755695956458041) > weight (0.25) 2021-03-09T08:14:54.276 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6071555617926262) > weight (0.25) 2021-03-09T08:14:54.276 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3387838816638332) > weight (0.25) 2021-03-09T08:14:54.277 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.33635160648793105) > weight (0.25) 2021-03-09T08:14:54.277 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T08:16:00.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8134198922715976) > weight (0.25) 2021-03-09T08:16:00.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.37499336623079127) > weight (0.25) 2021-03-09T08:16:00.280 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.43471357848297865) > weight (0.25) 2021-03-09T08:16:00.280 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8507630106762916) > weight (0.25) 2021-03-09T08:16:00.281 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T08:16:05.333 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9320315859763708) > weight (0.25) 2021-03-09T08:16:05.334 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7083365466385632) > weight (0.25) 2021-03-09T08:16:05.334 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4015758583032154) > weight (0.25) 2021-03-09T08:16:05.334 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6662572214928948) > weight (0.25) 2021-03-09T08:16:05.334 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T08:17:14.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.30142744886904027) > weight (0.25) 2021-03-09T08:17:14.389 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.37126048211923224) > weight (0.25) 2021-03-09T08:17:14.389 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7872685333939672) > weight (0.25) 2021-03-09T08:17:14.390 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5790702711132371) > weight (0.25) 2021-03-09T08:17:14.390 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T08:17:40.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5150540236209271) > weight (0.25) 2021-03-09T08:17:40.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8365929573771016) > weight (0.25) 2021-03-09T08:17:40.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8764512151234829) > weight (0.25) 2021-03-09T08:17:40.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3894706717833477) > weight (0.25) 2021-03-09T08:17:40.284 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T08:19:03.391 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7742611226906255) > weight (0.25) 2021-03-09T08:19:03.392 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9657505650758476) > weight (0.25) 2021-03-09T08:19:03.392 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6482145049483149) > weight (0.25) 2021-03-09T08:19:03.393 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T08:19:03.394 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T08:19:03.394 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T08:19:03.395 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T08:19:03.389+0000 7f78e38cd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 57465) UID: 1000 2021-03-09T08:19:03.595 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:19:08.396 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T08:19:08.396 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:19:08.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:19:08.397 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T08:19:08.398 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T08:19:08.398 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T08:19:08.401 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T08:19:08.401 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T08:19:35.284 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4255927005674125) > weight (0.25) 2021-03-09T08:19:35.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.34423745290832763) > weight (0.25) 2021-03-09T08:19:35.290 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5597205759126265) > weight (0.25) 2021-03-09T08:19:35.290 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T08:19:35.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T08:19:35.292 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T08:19:35.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T08:19:35.289+0000 7f7ca7290700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 52000) UID: 1000 2021-03-09T08:19:35.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:19:36.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T08:19:36.445+0000 7fc05f97c700 -1 rbd::mirror::ImageReplayer: 0x55bb899eb680 [3/08a00e86-016b-46cb-bca2-dcf2f528e189] start: already running 2021-03-09T08:19:36.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T08:19:36.445+0000 7fc05f97c700 -1 rbd::mirror::ImageReplayer: 0x55bb899eb900 [3/110cc5bb-34a2-4449-8efe-87f25e6218e2] start: already running 2021-03-09T08:19:36.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T08:19:36.453+0000 7fc05f97c700 -1 rbd::mirror::ImageReplayer: 0x55bb899eb680 [3/08a00e86-016b-46cb-bca2-dcf2f528e189] start: already running 2021-03-09T08:19:36.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T08:19:36.453+0000 7fc06017d700 -1 rbd::mirror::ImageReplayer: 0x55bb899eb900 [3/110cc5bb-34a2-4449-8efe-87f25e6218e2] start: already running 2021-03-09T08:19:36.462 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T08:19:36.453+0000 7fefd78a3700 -1 rbd::mirror::ImageReplayer: 0x5621c1d3ec80 [3/18a549b3-61f2-46e5-af4d-1a9b02c6e0ad] start: already running 2021-03-09T08:19:36.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T08:19:36.453+0000 7fefd78a3700 -1 rbd::mirror::ImageReplayer: 0x5621bdb65180 [3/1957e71d-23c8-464e-a815-0768c7c29be6] start: already running 2021-03-09T08:19:36.463 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T08:19:36.453+0000 7fefd78a3700 -1 rbd::mirror::ImageReplayer: 0x5621c1d3e000 [3/28f924e2-e881-4421-bb68-c0bf69c20856] start: already running 2021-03-09T08:19:36.468 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T08:19:36.459+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad0227cf0 2021-03-09T08:19:36.468 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T08:19:36.459+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad51074d0 2021-03-09T08:19:36.469 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T08:19:36.459+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9b979b0 2021-03-09T08:19:36.469 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T08:19:36.459+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9eb9e90 2021-03-09T08:19:36.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:19:36.465+0000 7fe50acdd700 -1 rbd::mirror::ImageReplayer: 0x557c9c883e00 [3/1a6951f1-4951-4047-8edb-1035961df315] start: already running 2021-03-09T08:19:36.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:19:36.465+0000 7fe50a4dc700 -1 rbd::mirror::ImageReplayer: 0x557c9c883680 [3/3b232a3b-57f6-42ac-8d1f-68890a58c5ed] start: already running 2021-03-09T08:19:36.472 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:19:36.465+0000 7fe50acdd700 -1 rbd::mirror::ImageReplayer: 0x557ca0a96300 [3/6cd921d3-989d-483b-ba40-00f209ebbd60] start: already running 2021-03-09T08:19:36.809 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:19:36.801+0000 7f082a074700 -1 reset not still connected to 0x55a99f963a00 2021-03-09T08:19:36.812 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:19:36.805+0000 7f082a074700 -1 reset not still connected to 0x55a9af4f88f0 2021-03-09T08:19:36.813 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:19:36.805+0000 7f082a074700 -1 reset not still connected to 0x55a9af4fbb50 2021-03-09T08:19:36.814 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:19:36.805+0000 7f082a074700 -1 reset not still connected to 0x55a9cf587810 2021-03-09T08:19:43.293 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T08:19:43.294 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:19:43.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:19:43.295 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T08:19:43.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T08:19:43.297 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T08:19:43.300 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T08:19:43.300 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T08:20:24.120 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T08:20:24.114+0000 7fdda4a8d700 -1 reset not still connected to 0x55aee4eea8a0 2021-03-09T08:20:24.121 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T08:20:24.114+0000 7fdda4a8d700 -1 reset not still connected to 0x55aefc559a80 2021-03-09T08:20:24.121 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T08:20:24.114+0000 7fdda4a8d700 -1 reset not still connected to 0x55af013067d0 2021-03-09T08:20:24.121 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T08:20:24.114+0000 7fdda4a8d700 -1 reset not still connected to 0x55af03a14490 2021-03-09T08:20:52.301 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T08:20:52.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T08:20:52.303 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T08:20:52.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:20:52.299+0000 7fe50b4de700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 58618) UID: 1000 2021-03-09T08:20:52.506 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:20:53.303 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T08:20:53.304 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:20:53.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:20:53.305 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T08:20:53.305 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T08:20:53.305 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T08:20:53.308 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T08:20:53.309 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T08:21:05.403 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:21:05.406 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:21:05.407 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T08:21:05.408 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:21:05.403+0000 7fada838a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 59395) UID: 1000 2021-03-09T08:21:05.608 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:21:13.409 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:21:13.410 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:21:13.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:21:13.411 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:21:13.412 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:21:13.412 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:21:13.415 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:21:13.415 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T08:22:30.310 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7886726470344372) > weight (0.25) 2021-03-09T08:22:30.342 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7741981279153899) > weight (0.25) 2021-03-09T08:22:30.343 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8728807884466888) > weight (0.25) 2021-03-09T08:22:30.343 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T08:22:30.344 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T08:22:30.344 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T08:22:30.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T08:22:30.338+0000 7fa0d3e57700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 59870) UID: 1000 2021-03-09T08:22:30.545 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:22:39.345 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T08:22:39.347 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:22:39.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:22:39.348 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T08:22:39.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T08:22:39.348 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T08:22:39.351 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T08:22:39.351 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-03-09T08:22:45.416 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:22:45.417 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:22:45.417 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T08:22:45.418 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:22:45.413+0000 7f9d9192a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 60070) UID: 1000 2021-03-09T08:22:45.618 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:22:54.418 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:22:54.420 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:22:54.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:22:54.421 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:22:54.422 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:22:54.422 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:22:54.425 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:22:54.426 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T08:23:58.353 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.31437993928312713) > weight (0.25) 2021-03-09T08:23:58.402 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.37729659089699263) > weight (0.25) 2021-03-09T08:23:58.402 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T08:23:58.403 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T08:23:58.403 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T08:23:58.404 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T08:23:58.400+0000 7fc06097e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 59579) UID: 1000 2021-03-09T08:23:58.604 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:24:07.404 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T08:24:07.405 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:24:07.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:24:07.406 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T08:24:07.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T08:24:07.406 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T08:24:07.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T08:24:07.415 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T08:24:09.595 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T08:24:09.591+0000 7fa08ad3c700 -1 reset not still connected to 0x564ace4038e0 2021-03-09T08:24:09.596 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T08:24:09.591+0000 7fa08ad3c700 -1 reset not still connected to 0x564acf277860 2021-03-09T08:24:09.597 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T08:24:09.591+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad51078e0 2021-03-09T08:24:09.597 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T08:24:09.591+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad5107cf0 2021-03-09T08:24:27.427 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5909735285557864) > weight (0.25) 2021-03-09T08:24:27.429 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7813683019089348) > weight (0.25) 2021-03-09T08:24:27.429 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6259034653990919) > weight (0.25) 2021-03-09T08:24:27.429 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.52750596157635) > weight (0.25) 2021-03-09T08:24:27.429 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T08:25:32.430 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4377270322463396) > weight (0.25) 2021-03-09T08:25:32.488 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T08:25:32.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T08:25:32.490 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T08:25:32.491 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T08:25:32.486+0000 7fbe943e5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 59487) UID: 1000 2021-03-09T08:25:32.691 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:25:40.491 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T08:25:40.494 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:25:40.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:25:40.495 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T08:25:40.496 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T08:25:40.497 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T08:25:40.500 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T08:25:40.501 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T08:26:02.416 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T08:26:02.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T08:26:02.442 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T08:26:02.442 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:26:02.438+0000 7fa9b47b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 59971) UID: 1000 2021-03-09T08:26:02.643 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:26:06.443 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T08:26:06.443 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:26:06.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:26:06.444 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T08:26:06.444 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T08:26:06.444 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T08:26:06.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T08:26:06.447 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T08:27:33.453 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6419061014203359) > weight (0.25) 2021-03-09T08:27:33.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6421288876670623) > weight (0.25) 2021-03-09T08:27:33.455 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T08:27:33.456 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T08:27:33.458 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T08:27:33.459 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T08:27:33.456+0000 7fccf67b9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 60351) UID: 1000 2021-03-09T08:27:33.659 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:27:34.458 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T08:27:34.459 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:27:34.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:27:34.459 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T08:27:34.460 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T08:27:34.461 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T08:27:34.464 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T08:27:34.465 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T08:27:35.502 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8358085630362366) > weight (0.25) 2021-03-09T08:27:35.502 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9227554602332472) > weight (0.25) 2021-03-09T08:27:35.502 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6231393651209087) > weight (0.25) 2021-03-09T08:27:35.503 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.912442625005881) > weight (0.25) 2021-03-09T08:27:35.503 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T08:28:40.739 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:28:40.735+0000 7f901a51c700 -1 reset not still connected to 0x5588fdfc34d0 2021-03-09T08:28:48.466 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T08:28:48.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T08:28:48.470 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T08:28:48.470 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T08:28:48.470 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:28:48.471 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:28:48.467+0000 7f32d8a93700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 60547) UID: 1000 2021-03-09T08:28:48.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:28:49.503 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9928684031040507) > weight (0.25) 2021-03-09T08:28:49.505 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T08:28:49.506 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T08:28:49.507 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T08:28:49.508 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T08:28:49.503+0000 7f6a55900700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 60456) UID: 1000 2021-03-09T08:28:49.709 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:28:53.509 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T08:28:53.510 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:28:53.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:28:53.511 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T08:28:53.511 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T08:28:53.512 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T08:28:53.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T08:28:53.519 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T08:28:54.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:28:54.573 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T08:28:54.573 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T08:28:54.574 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T08:28:54.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T08:28:54.580 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T08:29:53.519 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.880804364493078) > weight (0.25) 2021-03-09T08:29:53.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9271300752754579) > weight (0.25) 2021-03-09T08:29:53.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.27082072238672117) > weight (0.25) 2021-03-09T08:29:53.522 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5369014940648145) > weight (0.25) 2021-03-09T08:29:53.522 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T08:30:11.581 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T08:30:11.585 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T08:30:11.586 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T08:30:11.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:30:11.586+0000 7f07f5013700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 60836) UID: 1000 2021-03-09T08:30:11.791 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:30:13.587 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T08:30:13.588 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:30:13.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:30:13.589 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T08:30:13.590 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T08:30:13.592 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T08:30:13.595 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T08:30:13.596 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T08:31:03.523 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:31:03.571 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:31:03.572 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T08:31:03.573 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:31:03.569+0000 7f633505a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 60260) UID: 1000 2021-03-09T08:31:03.772 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:31:08.573 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:31:08.577 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:31:08.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:31:08.578 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:31:08.578 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:31:08.579 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:31:08.581 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:31:08.581 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T08:31:14.597 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7135891968329342) > weight (0.25) 2021-03-09T08:31:14.603 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8448545212179481) > weight (0.25) 2021-03-09T08:31:14.603 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.522199708483099) > weight (0.25) 2021-03-09T08:31:14.604 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5670160009381012) > weight (0.25) 2021-03-09T08:31:14.604 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T08:32:14.582 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:32:14.585 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:32:14.587 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T08:32:14.589 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:32:14.583+0000 7f566f11c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 61038) UID: 1000 2021-03-09T08:32:14.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:32:20.588 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:32:20.596 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:32:20.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:32:20.597 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:32:20.597 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:32:20.598 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:32:20.601 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:32:20.601 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-03-09T08:33:03.605 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6152649316119947) > weight (0.25) 2021-03-09T08:33:03.793 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T08:33:03.794 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T08:33:03.795 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T08:33:03.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T08:33:03.791+0000 7fefd88a5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 59296) UID: 1000 2021-03-09T08:33:03.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:33:04.796 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T08:33:04.797 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:33:04.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:33:04.798 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T08:33:04.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T08:33:04.799 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T08:33:04.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T08:33:04.806 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T08:33:35.602 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:33:35.604 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:33:35.605 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T08:33:35.606 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:33:35.602+0000 7fcd84569700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 61130) UID: 1000 2021-03-09T08:33:35.806 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:33:40.606 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:33:40.609 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:33:40.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:33:40.610 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:33:40.610 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:33:40.610 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:33:40.613 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:33:40.613 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T08:34:16.807 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2926640620680223) > weight (0.25) 2021-03-09T08:34:16.808 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7198081542008666) > weight (0.25) 2021-03-09T08:34:16.808 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9020286002831905) > weight (0.25) 2021-03-09T08:34:16.808 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T08:34:16.809 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T08:34:16.809 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T08:34:16.810 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T08:34:16.805+0000 7fcf4731f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 60161) UID: 1000 2021-03-09T08:34:17.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:34:17.810 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T08:34:17.811 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:34:17.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:34:17.811 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T08:34:17.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T08:34:17.812 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T08:34:17.815 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T08:34:17.815 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-03-09T08:35:09.615 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3297291723960012) > weight (0.25) 2021-03-09T08:35:10.730 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3860260507965647) > weight (0.25) 2021-03-09T08:35:10.732 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4671137154571249) > weight (0.25) 2021-03-09T08:35:10.733 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4987612163837276) > weight (0.25) 2021-03-09T08:35:10.733 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T08:35:32.816 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9610132363219309) > weight (0.25) 2021-03-09T08:35:32.816 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T08:35:32.817 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T08:35:32.818 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T08:35:32.819 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T08:35:32.816+0000 7fc5b58fe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 61221) UID: 1000 2021-03-09T08:35:33.019 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:35:37.819 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T08:35:37.819 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:35:37.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:35:37.820 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T08:35:37.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T08:35:37.821 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T08:35:37.823 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T08:35:37.824 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T08:36:43.825 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T08:36:43.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T08:36:43.856 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T08:36:43.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:36:43.855+0000 7f3ae0c89700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 60938) UID: 1000 2021-03-09T08:36:44.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:36:51.857 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T08:36:51.858 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:36:51.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:36:51.859 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T08:36:51.859 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T08:36:51.859 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T08:36:51.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T08:36:51.863 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T08:36:53.735 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9166785742922242) > weight (0.25) 2021-03-09T08:36:53.736 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.44726438107286015) > weight (0.25) 2021-03-09T08:36:53.736 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8013142977449249) > weight (0.25) 2021-03-09T08:36:53.736 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9190906312678386) > weight (0.25) 2021-03-09T08:36:53.736 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T08:38:10.738 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7942449740088008) > weight (0.25) 2021-03-09T08:38:10.751 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.842470734488351) > weight (0.25) 2021-03-09T08:38:10.752 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T08:38:10.753 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T08:38:10.753 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T08:38:10.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T08:38:10.753+0000 7f4555a84700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 59678) UID: 1000 2021-03-09T08:38:10.956 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:38:15.754 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T08:38:15.755 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:38:15.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:38:15.755 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T08:38:15.756 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T08:38:15.756 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T08:38:15.759 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T08:38:15.759 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T08:38:28.864 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6704006707974891) > weight (0.25) 2021-03-09T08:38:28.866 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5474211607688443) > weight (0.25) 2021-03-09T08:38:28.866 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6915603545490011) > weight (0.25) 2021-03-09T08:38:28.866 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3802556578020405) > weight (0.25) 2021-03-09T08:38:28.867 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-03-09T08:38:38.362 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:38:38.358+0000 7f901a51c700 -1 reset not still connected to 0x5588fb0ba9c0 2021-03-09T08:38:38.362 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:38:38.358+0000 7f901a51c700 -1 reset not still connected to 0x5588fb0bab60 2021-03-09T08:38:38.362 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:38:38.358+0000 7f901a51c700 -1 reset not still connected to 0x5588fb0bb1e0 2021-03-09T08:38:38.362 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:38:38.358+0000 7f901a51c700 -1 reset not still connected to 0x5588fb0bc2f0 2021-03-09T08:38:38.363 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:38:38.358+0000 7f901a51c700 -1 reset not still connected to 0x5588fb0bd810 2021-03-09T08:38:38.363 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:38:38.358+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf00d0 2021-03-09T08:39:20.760 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4786054511286746) > weight (0.25) 2021-03-09T08:39:20.764 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8009858626425499) > weight (0.25) 2021-03-09T08:39:20.764 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8239004126555148) > weight (0.25) 2021-03-09T08:39:20.764 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5425858197432333) > weight (0.25) 2021-03-09T08:39:20.765 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-03-09T08:40:06.868 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7949563529460506) > weight (0.25) 2021-03-09T08:40:06.869 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.42881829343548605) > weight (0.25) 2021-03-09T08:40:06.869 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.37355598113205823) > weight (0.25) 2021-03-09T08:40:06.869 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T08:40:06.870 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T08:40:06.870 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T08:40:06.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T08:40:06.867+0000 7f60e319e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 61411) UID: 1000 2021-03-09T08:40:07.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:40:10.872 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T08:40:10.872 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:40:10.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:40:10.873 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T08:40:10.873 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T08:40:10.874 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T08:40:10.876 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T08:40:10.877 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T08:40:38.649 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:40:38.646+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf2be0 2021-03-09T08:40:38.650 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:40:38.646+0000 7f901a51c700 -1 reset not still connected to 0x558904593400 2021-03-09T08:40:39.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.28499353234020974) > weight (0.25) 2021-03-09T08:40:39.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9057760398282907) > weight (0.25) 2021-03-09T08:40:39.766 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3835134659427838) > weight (0.25) 2021-03-09T08:40:39.767 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7516928950182938) > weight (0.25) 2021-03-09T08:40:39.767 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T08:41:08.653 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:41:08.645+0000 7f901a51c700 -1 reset not still connected to 0x5588ffaf2be0 2021-03-09T08:41:41.878 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5163500367990931) > weight (0.25) 2021-03-09T08:41:41.880 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9793849578334198) > weight (0.25) 2021-03-09T08:41:41.880 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.44190036453963477) > weight (0.25) 2021-03-09T08:41:41.881 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9435052023921916) > weight (0.25) 2021-03-09T08:41:41.881 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T08:42:12.768 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9746694778394944) > weight (0.25) 2021-03-09T08:42:12.770 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9735437858930717) > weight (0.25) 2021-03-09T08:42:12.770 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5540057966920268) > weight (0.25) 2021-03-09T08:42:12.770 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7494649837766205) > weight (0.25) 2021-03-09T08:42:12.771 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T08:43:06.883 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.697858925910029) > weight (0.25) 2021-03-09T08:43:06.885 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9849132243414898) > weight (0.25) 2021-03-09T08:43:06.885 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6179799326553531) > weight (0.25) 2021-03-09T08:43:06.885 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5741452048617105) > weight (0.25) 2021-03-09T08:43:06.885 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T08:43:13.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.35611449306529186) > weight (0.25) 2021-03-09T08:43:13.772 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7302252401336974) > weight (0.25) 2021-03-09T08:43:13.772 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.40362569980176954) > weight (0.25) 2021-03-09T08:43:13.773 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T08:43:13.774 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T08:43:13.774 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T08:43:13.775 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T08:43:13.772+0000 7f82716e2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 59777) UID: 1000 2021-03-09T08:43:13.975 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:43:17.775 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T08:43:17.775 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:43:17.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:43:17.776 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T08:43:17.777 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T08:43:17.777 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T08:43:17.780 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T08:43:17.780 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-03-09T08:44:15.886 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T08:44:15.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T08:44:15.888 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T08:44:15.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:44:15.883+0000 7fc742bc8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 61615) UID: 1000 2021-03-09T08:44:16.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:44:17.889 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T08:44:17.889 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:44:17.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:44:17.890 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T08:44:17.890 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T08:44:17.891 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T08:44:17.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T08:44:17.894 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-03-09T08:44:53.781 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6293518865619355) > weight (0.25) 2021-03-09T08:44:53.782 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6692968319008631) > weight (0.25) 2021-03-09T08:44:53.782 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.881988513966062) > weight (0.25) 2021-03-09T08:44:53.783 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9033847306466384) > weight (0.25) 2021-03-09T08:44:53.783 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T08:45:25.895 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T08:45:25.896 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T08:45:25.897 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T08:45:25.897 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:45:25.894+0000 7f3ebe95c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 62002) UID: 1000 2021-03-09T08:45:26.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:45:31.898 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T08:45:31.898 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:45:31.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:45:31.899 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T08:45:31.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T08:45:31.900 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T08:45:31.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T08:45:31.903 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-03-09T08:45:53.784 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9137595312575335) > weight (0.25) 2021-03-09T08:45:53.786 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T08:45:53.787 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T08:45:53.788 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T08:45:53.789 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T08:45:53.781+0000 7f68b8e98700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 60745) UID: 1000 2021-03-09T08:45:53.988 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:45:54.789 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T08:45:54.789 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:45:54.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:45:54.790 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T08:45:54.790 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T08:45:54.791 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T08:45:54.793 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T08:45:54.794 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T08:46:57.904 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.532705506766069) > weight (0.25) 2021-03-09T08:46:57.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9655274138394306) > weight (0.25) 2021-03-09T08:46:57.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.720263915638113) > weight (0.25) 2021-03-09T08:46:57.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3114342295704192) > weight (0.25) 2021-03-09T08:46:57.907 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T08:47:35.794 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:47:35.796 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:47:35.796 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T08:47:35.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:47:35.796+0000 7f8b77cbf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 61320) UID: 1000 2021-03-09T08:47:35.997 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:47:41.797 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:47:41.798 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:47:41.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:47:41.799 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:47:41.799 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:47:41.799 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:47:41.802 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:47:41.803 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-03-09T08:48:11.260 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:48:11.253+0000 7f901a51c700 -1 reset not still connected to 0x55890173b450 2021-03-09T08:48:11.260 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:48:11.253+0000 7f901a51c700 -1 reset not still connected to 0x55890173d0c0 2021-03-09T08:48:11.261 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:48:11.253+0000 7f901a51c700 -1 reset not still connected to 0x558903e2bad0 2021-03-09T08:48:45.908 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8524301490735502) > weight (0.25) 2021-03-09T08:48:45.908 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3684703816215822) > weight (0.25) 2021-03-09T08:48:45.909 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4919773346809344) > weight (0.25) 2021-03-09T08:48:45.909 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9705880831191894) > weight (0.25) 2021-03-09T08:48:45.909 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T08:49:07.804 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:49:07.805 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:49:07.806 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T08:49:07.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:49:07.802+0000 7f9bb1343700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62297) UID: 1000 2021-03-09T08:49:09.008 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:49:15.807 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:49:15.807 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:49:15.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:49:15.808 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:49:15.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:49:15.809 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:49:15.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:49:15.812 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-03-09T08:49:45.911 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7703204079461627) > weight (0.25) 2021-03-09T08:49:45.912 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T08:49:45.913 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T08:49:45.913 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T08:49:45.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T08:49:45.909+0000 7f155e677700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 61515) UID: 1000 2021-03-09T08:49:46.118 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:49:48.914 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T08:49:48.916 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:49:48.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:49:48.917 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T08:49:48.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T08:49:48.918 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T08:49:48.921 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T08:49:48.921 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T08:50:14.838 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:50:14.833+0000 7f901a51c700 -1 reset not still connected to 0x55890173b450 2021-03-09T08:50:14.838 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:50:14.833+0000 7f901a51c700 -1 reset not still connected to 0x55890173d0c0 2021-03-09T08:50:14.838 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:50:14.833+0000 7f901a51c700 -1 reset not still connected to 0x558903e2bad0 2021-03-09T08:50:34.814 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.914947292330185) > weight (0.25) 2021-03-09T08:50:34.816 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T08:50:34.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T08:50:34.817 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T08:50:34.817 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T08:50:34.816+0000 7f696949d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 62203) UID: 1000 2021-03-09T08:50:35.018 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:50:41.817 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T08:50:41.818 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:50:41.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:50:41.819 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T08:50:41.819 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T08:50:41.820 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T08:50:41.822 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T08:50:41.823 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T08:51:05.922 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.26109713242306964) > weight (0.25) 2021-03-09T08:51:05.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9971915374881174) > weight (0.25) 2021-03-09T08:51:05.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7052285430673235) > weight (0.25) 2021-03-09T08:51:05.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.3254037692137295) > weight (0.25) 2021-03-09T08:51:05.925 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T08:52:10.824 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:52:10.836 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:52:10.838 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T08:52:10.839 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:52:10.835+0000 7f62b036f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62388) UID: 1000 2021-03-09T08:52:11.039 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:52:19.839 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:52:19.842 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:52:19.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:52:19.843 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:52:19.843 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:52:19.844 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:52:19.847 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:52:19.847 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-03-09T08:53:00.927 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4327335443935152) > weight (0.25) 2021-03-09T08:53:00.929 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T08:53:00.929 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T08:53:00.930 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T08:53:00.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T08:53:00.926+0000 7f271eda0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 62479) UID: 1000 2021-03-09T08:53:01.131 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:53:03.930 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T08:53:03.932 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:53:03.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:53:03.933 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T08:53:03.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T08:53:03.935 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T08:53:03.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T08:53:03.939 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T08:53:45.848 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T08:53:45.904 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T08:53:45.905 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T08:53:45.906 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T08:53:45.906 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:53:45.907 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T08:53:45.901+0000 7f8d75b60700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62672) UID: 1000 2021-03-09T08:53:46.106 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:53:47.752 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T08:53:47.747+0000 7fdda4a8d700 -1 reset not still connected to 0x55af004b9c70 2021-03-09T08:53:47.755 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T08:53:47.751+0000 7fdda4a8d700 -1 reset not still connected to 0x55af0246b740 2021-03-09T08:53:52.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:53:52.008 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T08:53:52.009 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T08:53:52.009 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T08:53:52.013 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T08:53:52.014 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T08:54:35.940 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T08:54:35.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T08:54:35.943 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T08:54:35.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T08:54:35.940+0000 7f7073584700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 62104) UID: 1000 2021-03-09T08:54:36.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:54:39.944 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T08:54:39.945 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:54:39.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:54:39.946 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T08:54:39.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T08:54:39.947 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T08:54:39.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T08:54:39.957 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T08:55:04.989 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T08:55:04.981+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad5b04560 2021-03-09T08:55:04.989 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T08:55:04.981+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9b95110 2021-03-09T08:55:04.990 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T08:55:04.985+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9eb7ee0 2021-03-09T08:55:37.015 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5561971209186696) > weight (0.25) 2021-03-09T08:55:37.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6979846762918303) > weight (0.25) 2021-03-09T08:55:37.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9134596353478015) > weight (0.25) 2021-03-09T08:55:37.016 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T08:55:37.017 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T08:55:37.017 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T08:55:37.018 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T08:55:37.015+0000 7f8420947700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 61911) UID: 1000 2021-03-09T08:55:37.218 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:55:43.018 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T08:55:43.019 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:55:43.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:55:43.020 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T08:55:43.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T08:55:43.021 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T08:55:43.023 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T08:55:43.024 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T08:55:43.958 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4044245755846112) > weight (0.25) 2021-03-09T08:55:43.958 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3195878871664072) > weight (0.25) 2021-03-09T08:55:43.959 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5525911189997753) > weight (0.25) 2021-03-09T08:55:43.959 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7432761634025359) > weight (0.25) 2021-03-09T08:55:43.959 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T08:56:52.253 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:56:52.246+0000 7f082a074700 -1 reset not still connected to 0x55a9a3819ad0 2021-03-09T08:56:52.254 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T08:56:52.246+0000 7f082a074700 -1 reset not still connected to 0x55a9a3819d40 2021-03-09T08:57:36.024 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2891932556304223) > weight (0.25) 2021-03-09T08:57:36.055 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4710710479749627) > weight (0.25) 2021-03-09T08:57:36.056 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.752025416704294) > weight (0.25) 2021-03-09T08:57:36.056 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T08:57:36.057 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T08:57:36.057 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T08:57:36.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T08:57:36.053+0000 7fb6920eb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 63057) UID: 1000 2021-03-09T08:57:36.258 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:57:39.058 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T08:57:39.058 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:57:39.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:57:39.059 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T08:57:39.059 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T08:57:39.060 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T08:57:39.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T08:57:39.063 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T08:57:40.960 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2987408057230755) > weight (0.25) 2021-03-09T08:57:40.960 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9157090422708785) > weight (0.25) 2021-03-09T08:57:40.961 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T08:57:40.961 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T08:57:40.962 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T08:57:40.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T08:57:40.957+0000 7fe3c0226700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 60646) UID: 1000 2021-03-09T08:57:41.363 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:57:45.964 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T08:57:45.965 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:57:45.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:57:45.966 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T08:57:45.967 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T08:57:45.968 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T08:57:45.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T08:57:45.978 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T08:58:14.263 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:58:14.251+0000 7f901a51c700 -1 reset not still connected to 0x558903e2c220 2021-03-09T08:58:14.264 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T08:58:14.251+0000 7f901a51c700 -1 reset not still connected to 0x558904ab1860 2021-03-09T08:59:26.065 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7285790724255695) > weight (0.25) 2021-03-09T08:59:26.068 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5400024948381065) > weight (0.25) 2021-03-09T08:59:26.069 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T08:59:26.069 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T08:59:26.070 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T08:59:26.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T08:59:26.067+0000 7f2c39aab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 61715) UID: 1000 2021-03-09T08:59:26.271 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:59:27.223 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T08:59:27.219+0000 7ff6cec20700 -1 rbd::mirror::ImageReplayer: 0x561353258c80 [3/18a549b3-61f2-46e5-af4d-1a9b02c6e0ad] start: already running 2021-03-09T08:59:27.233 INFO:tasks.ceph.cluster2.osd.0.smithi144.stderr:2021-03-09T08:59:27.227+0000 7f0e42124700 -1 reset not still connected to 0x5573f2a032b0 2021-03-09T08:59:31.070 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T08:59:31.073 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:59:31.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:59:31.073 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T08:59:31.074 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T08:59:31.074 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T08:59:31.077 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T08:59:31.077 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T08:59:37.979 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3343274480152779) > weight (0.25) 2021-03-09T08:59:37.980 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T08:59:37.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T08:59:37.980 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T08:59:37.981 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T08:59:37.981 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T08:59:37.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T08:59:37.979+0000 7f0f054fc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 62764) UID: 1000 2021-03-09T08:59:38.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T08:59:44.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T08:59:44.083 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T08:59:44.083 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T08:59:44.084 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T08:59:44.086 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T08:59:44.087 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-03-09T09:00:58.078 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T09:00:58.082 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T09:00:58.083 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T09:00:58.084 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T09:00:58.078+0000 7f80f2fad700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 62863) UID: 1000 2021-03-09T09:00:58.283 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:01:00.084 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T09:01:00.085 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:01:00.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:01:00.086 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T09:01:00.087 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T09:01:00.087 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T09:01:00.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T09:01:00.091 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-03-09T09:01:42.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3224637255103471) > weight (0.25) 2021-03-09T09:01:42.088 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5480771646144139) > weight (0.25) 2021-03-09T09:01:42.088 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T09:01:42.089 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T09:01:42.090 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T09:01:42.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T09:01:42.089+0000 7fa883836700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 63241) UID: 1000 2021-03-09T09:01:42.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:01:44.091 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T09:01:44.092 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:01:44.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:01:44.093 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T09:01:44.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T09:01:44.094 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T09:01:44.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T09:01:44.098 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T09:02:58.092 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9344467204548034) > weight (0.25) 2021-03-09T09:02:58.093 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6026249185379067) > weight (0.25) 2021-03-09T09:02:58.094 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T09:02:58.095 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T09:02:58.095 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T09:02:58.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T09:02:58.091+0000 7f441f123700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 63342) UID: 1000 2021-03-09T09:02:58.149 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T09:02:58.143+0000 7fc0e0d45700 -1 reset not still connected to 0x55e8d84bd860 2021-03-09T09:02:58.297 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:03:04.096 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T09:03:04.097 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:03:04.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:03:04.098 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T09:03:04.098 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T09:03:04.098 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T09:03:04.101 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T09:03:04.102 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T09:03:23.099 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2953844874909831) > weight (0.25) 2021-03-09T09:03:23.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.723264993116819) > weight (0.25) 2021-03-09T09:03:23.100 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T09:03:23.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T09:03:23.101 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T09:03:23.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T09:03:23.099+0000 7fa10dd95700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 63625) UID: 1000 2021-03-09T09:03:23.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:03:27.102 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T09:03:27.102 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:03:27.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:03:27.103 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T09:03:27.103 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T09:03:27.104 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T09:03:27.107 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T09:03:27.107 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T09:04:04.104 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3196120838883927) > weight (0.25) 2021-03-09T09:04:04.104 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T09:04:04.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T09:04:04.106 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T09:04:04.106 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T09:04:04.102+0000 7f3b15951700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 62579) UID: 1000 2021-03-09T09:04:04.307 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:04:07.107 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T09:04:07.107 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:04:07.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:04:07.108 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T09:04:07.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T09:04:07.109 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T09:04:07.111 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T09:04:07.112 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T09:04:37.109 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5643146797999791) > weight (0.25) 2021-03-09T09:04:37.109 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9256868959903324) > weight (0.25) 2021-03-09T09:04:37.110 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.36288613984663853) > weight (0.25) 2021-03-09T09:04:37.110 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7289207235391436) > weight (0.25) 2021-03-09T09:04:37.110 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T09:05:08.113 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8627785946917259) > weight (0.25) 2021-03-09T09:05:08.116 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7899760571167674) > weight (0.25) 2021-03-09T09:05:08.116 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.577793724632056) > weight (0.25) 2021-03-09T09:05:08.116 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6219614110598476) > weight (0.25) 2021-03-09T09:05:08.117 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T09:05:54.112 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7494538355767897) > weight (0.25) 2021-03-09T09:05:54.191 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7227265370660151) > weight (0.25) 2021-03-09T09:05:54.192 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7318381098088984) > weight (0.25) 2021-03-09T09:05:54.192 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5562156570429807) > weight (0.25) 2021-03-09T09:05:54.192 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T09:06:10.117 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9351241854966987) > weight (0.25) 2021-03-09T09:06:10.121 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25427695757346624) > weight (0.25) 2021-03-09T09:06:10.121 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8987604810772706) > weight (0.25) 2021-03-09T09:06:10.121 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7878085435134011) > weight (0.25) 2021-03-09T09:06:10.122 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T09:06:39.305 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:06:39.297+0000 7f901a51c700 -1 reset not still connected to 0x55890173dc20 2021-03-09T09:07:20.123 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T09:07:20.126 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T09:07:20.126 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T09:07:20.126 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T09:07:20.127 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:07:20.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T09:07:20.123+0000 7fa60435d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 63533) UID: 1000 2021-03-09T09:07:20.327 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:07:23.193 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.47098378258157014) > weight (0.25) 2021-03-09T09:07:23.194 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5348810204266379) > weight (0.25) 2021-03-09T09:07:23.194 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7658239996589582) > weight (0.25) 2021-03-09T09:07:23.194 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6984303541161941) > weight (0.25) 2021-03-09T09:07:23.195 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T09:07:26.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:07:26.229 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T09:07:26.229 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T09:07:26.230 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T09:07:26.233 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T09:07:26.233 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T09:08:31.234 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9519068483695623) > weight (0.25) 2021-03-09T09:08:31.237 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T09:08:31.238 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T09:08:31.239 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T09:08:31.239 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T09:08:31.238+0000 7f5ca1ed5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 63915) UID: 1000 2021-03-09T09:08:31.440 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:08:37.239 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T09:08:37.240 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:08:37.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:08:37.240 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T09:08:37.241 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T09:08:37.241 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T09:08:37.244 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T09:08:37.244 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T09:09:09.196 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:09:09.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:09:09.197 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T09:09:09.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:09:09.193+0000 7f1ff3589700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 62955) UID: 1000 2021-03-09T09:09:09.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:09:13.199 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:09:13.199 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:09:13.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:09:13.200 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:09:13.200 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:09:13.201 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:09:13.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:09:13.204 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T09:10:08.245 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7648059403780034) > weight (0.25) 2021-03-09T09:10:08.246 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6175259891779653) > weight (0.25) 2021-03-09T09:10:08.247 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.26467549139810553) > weight (0.25) 2021-03-09T09:10:08.247 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.47718313023856784) > weight (0.25) 2021-03-09T09:10:08.247 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T09:10:26.205 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:10:26.208 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:10:26.208 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T09:10:26.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:10:26.204+0000 7f782d087700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64193) UID: 1000 2021-03-09T09:10:26.410 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:10:33.209 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:10:33.209 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:10:33.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:10:33.210 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:10:33.211 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:10:33.211 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:10:33.214 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:10:33.215 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T09:10:37.489 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:10:37.486+0000 7f901a51c700 -1 reset not still connected to 0x558904ab1450 2021-03-09T09:11:35.215 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:11:35.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:11:35.220 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T09:11:35.220 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:11:35.218+0000 7f6100dc1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64295) UID: 1000 2021-03-09T09:11:35.421 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:11:42.221 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:11:42.222 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:11:42.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:11:42.223 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:11:42.223 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:11:42.223 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:11:42.226 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:11:42.227 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T09:11:54.249 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8277253917941423) > weight (0.25) 2021-03-09T09:11:54.251 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6363490529135241) > weight (0.25) 2021-03-09T09:11:54.251 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T09:11:54.251 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T09:11:54.252 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T09:11:54.253 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T09:11:54.246+0000 7fa830117700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 63725) UID: 1000 2021-03-09T09:11:54.453 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:11:57.253 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T09:11:57.254 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:11:57.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:11:57.255 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T09:11:57.256 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T09:11:57.256 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T09:11:57.260 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T09:11:57.260 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T09:12:07.496 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:12:07.491+0000 7f901a51c700 -1 reset not still connected to 0x558904ab1450 2021-03-09T09:12:16.764 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T09:12:16.758+0000 7fc0e0d45700 -1 reset not still connected to 0x55e8dc373ad0 2021-03-09T09:12:55.228 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:12:55.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:12:55.231 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T09:12:55.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:12:55.225+0000 7fa966994700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64395) UID: 1000 2021-03-09T09:12:55.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:12:56.232 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:12:56.232 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:12:56.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:12:56.233 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:12:56.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:12:56.234 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:12:56.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:12:56.239 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T09:13:24.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5583303987397811) > weight (0.25) 2021-03-09T09:13:24.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6296248758971064) > weight (0.25) 2021-03-09T09:13:24.263 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.42205072616822803) > weight (0.25) 2021-03-09T09:13:24.263 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7459382987395293) > weight (0.25) 2021-03-09T09:13:24.263 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T09:14:20.240 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.563841793782644) > weight (0.25) 2021-03-09T09:14:20.243 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.836975807663563) > weight (0.25) 2021-03-09T09:14:20.244 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8781355251996888) > weight (0.25) 2021-03-09T09:14:20.244 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T09:14:20.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T09:14:20.245 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T09:14:20.246 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:14:20.244+0000 7fc576e57700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 61807) UID: 1000 2021-03-09T09:14:20.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:14:26.247 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T09:14:26.247 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:14:26.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:14:26.248 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T09:14:26.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T09:14:26.249 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T09:14:26.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T09:14:26.252 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T09:14:26.464 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:14:26.459+0000 7f901a51c700 -1 reset not still connected to 0x5588f277fb50 2021-03-09T09:14:26.465 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:14:26.459+0000 7f901a51c700 -1 reset not still connected to 0x5589045912b0 2021-03-09T09:14:26.469 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:14:26.467+0000 7f901a51c700 -1 reset not still connected to 0x558904ab2f20 2021-03-09T09:15:23.264 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9766982550860173) > weight (0.25) 2021-03-09T09:15:23.266 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7764464333087273) > weight (0.25) 2021-03-09T09:15:23.266 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7157261143180824) > weight (0.25) 2021-03-09T09:15:23.266 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T09:15:23.267 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T09:15:23.269 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T09:15:23.269 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T09:15:23.262+0000 7ff6cfc22700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 63150) UID: 1000 2021-03-09T09:15:23.469 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:15:28.269 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T09:15:28.271 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:15:28.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:15:28.272 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T09:15:28.273 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T09:15:28.274 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T09:15:28.277 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T09:15:28.277 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-03-09T09:15:40.253 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:15:40.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:15:40.254 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T09:15:40.255 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:15:40.250+0000 7f8110a5e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64586) UID: 1000 2021-03-09T09:15:40.455 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:15:41.255 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:15:41.257 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:15:41.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:15:41.257 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:15:41.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:15:41.258 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:15:41.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:15:41.261 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-03-09T09:16:49.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7518057959048705) > weight (0.25) 2021-03-09T09:16:49.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.752507110213535) > weight (0.25) 2021-03-09T09:16:49.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3523335699845056) > weight (0.25) 2021-03-09T09:16:49.283 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T09:16:49.284 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T09:16:49.284 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T09:16:49.285 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T09:16:49.277+0000 7fe9ceed6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 64790) UID: 1000 2021-03-09T09:16:49.486 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:16:51.285 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T09:16:51.285 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:16:51.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:16:51.286 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T09:16:51.287 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T09:16:51.287 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T09:16:51.293 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T09:16:51.293 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T09:17:17.263 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:17:17.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:17:17.264 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T09:17:17.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:17:17.260+0000 7f947afb4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 64881) UID: 1000 2021-03-09T09:17:17.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:17:19.265 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:17:19.265 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:17:19.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:17:19.266 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:17:19.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:17:19.271 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:17:19.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:17:19.274 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T09:17:23.602 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T09:17:23.594+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9eb7e10 2021-03-09T09:17:23.602 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T09:17:23.594+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9eb8630 2021-03-09T09:17:23.603 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T09:17:23.594+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9eb98e0 2021-03-09T09:17:37.487 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T09:17:37.484+0000 7f082a074700 -1 reset not still connected to 0x55a990d33040 2021-03-09T09:17:37.648 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T09:17:37.484+0000 7f082a074700 -1 reset not still connected to 0x55a998838b60 2021-03-09T09:18:25.294 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T09:18:25.299 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T09:18:25.300 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T09:18:25.301 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T09:18:25.295+0000 7fe874040700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 64011) UID: 1000 2021-03-09T09:18:25.501 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:18:30.302 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T09:18:30.302 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:18:30.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:18:30.303 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T09:18:30.303 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T09:18:30.304 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T09:18:30.306 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T09:18:30.307 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T09:18:36.581 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:18:36.572+0000 7f901a51c700 -1 reset not still connected to 0x5588fb9474d0 2021-03-09T09:18:36.582 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:18:36.572+0000 7f901a51c700 -1 reset not still connected to 0x55890173de90 2021-03-09T09:18:36.582 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:18:36.572+0000 7f901a51c700 -1 reset not still connected to 0x5589045905b0 2021-03-09T09:18:36.589 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:18:36.580+0000 7f901a51c700 -1 reset not still connected to 0x558904592220 2021-03-09T09:18:36.589 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:18:36.580+0000 7f901a51c700 -1 reset not still connected to 0x5589045930c0 2021-03-09T09:18:36.590 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:18:36.580+0000 7f901a51c700 -1 reset not still connected to 0x558904593dc0 2021-03-09T09:19:18.275 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9846458940593731) > weight (0.25) 2021-03-09T09:19:18.277 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4001967516178794) > weight (0.25) 2021-03-09T09:19:18.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8232440543011531) > weight (0.25) 2021-03-09T09:19:18.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9730960810229405) > weight (0.25) 2021-03-09T09:19:18.278 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T09:20:01.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5071739955090874) > weight (0.25) 2021-03-09T09:20:01.309 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5840967630400937) > weight (0.25) 2021-03-09T09:20:01.310 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6297013529152033) > weight (0.25) 2021-03-09T09:20:01.310 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.30086519633144404) > weight (0.25) 2021-03-09T09:20:01.311 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T09:20:43.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5626217698241186) > weight (0.25) 2021-03-09T09:20:43.282 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T09:20:43.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T09:20:43.283 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T09:20:43.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:20:43.277+0000 7f1fd680a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 63433) UID: 1000 2021-03-09T09:20:43.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:20:45.285 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T09:20:45.285 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:20:45.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:20:45.286 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T09:20:45.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T09:20:45.287 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T09:20:45.290 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T09:20:45.290 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T09:20:51.704 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:20:51.696+0000 7f901a51c700 -1 reset not still connected to 0x5588fa398dd0 2021-03-09T09:20:51.705 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:20:51.696+0000 7f901a51c700 -1 reset not still connected to 0x5588fb944410 2021-03-09T09:20:51.705 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:20:51.696+0000 7f901a51c700 -1 reset not still connected to 0x5588fb9474d0 2021-03-09T09:20:51.705 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:20:51.696+0000 7f901a51c700 -1 reset not still connected to 0x55890173de90 2021-03-09T09:20:51.705 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:20:51.696+0000 7f901a51c700 -1 reset not still connected to 0x5589045905b0 2021-03-09T09:20:51.708 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:20:51.700+0000 7f901a51c700 -1 reset not still connected to 0x558904592220 2021-03-09T09:20:51.712 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:20:51.704+0000 7f901a51c700 -1 reset not still connected to 0x5589045930c0 2021-03-09T09:20:51.712 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:20:51.704+0000 7f901a51c700 -1 reset not still connected to 0x558904593dc0 2021-03-09T09:21:07.494 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:21:07.487+0000 7f901a51c700 -1 reset not still connected to 0x558904ab1450 2021-03-09T09:21:25.311 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8006458802089255) > weight (0.25) 2021-03-09T09:21:25.315 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5469851838573341) > weight (0.25) 2021-03-09T09:21:25.315 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.997686207909182) > weight (0.25) 2021-03-09T09:21:25.316 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T09:21:25.316 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T09:21:25.317 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T09:21:25.318 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T09:21:25.312+0000 7fe06b869700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 64981) UID: 1000 2021-03-09T09:21:25.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:21:30.317 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T09:21:30.318 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:21:30.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:21:30.318 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T09:21:30.319 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T09:21:30.319 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T09:21:30.322 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T09:21:30.322 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T09:22:19.291 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:22:19.295 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:22:19.295 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T09:22:19.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:22:19.291+0000 7fc859879700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 65075) UID: 1000 2021-03-09T09:22:19.497 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:22:27.297 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:22:27.297 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:22:27.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:22:27.298 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:22:27.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:22:27.299 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:22:27.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:22:27.302 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T09:22:52.323 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T09:22:52.324 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T09:22:52.325 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T09:22:52.326 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T09:22:52.319+0000 7f00eac88700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 65174) UID: 1000 2021-03-09T09:22:52.526 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:22:53.326 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T09:22:53.327 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:22:53.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:22:53.327 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T09:22:53.328 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T09:22:53.328 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T09:22:53.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T09:22:53.331 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T09:24:22.303 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:24:22.345 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:24:22.346 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T09:24:22.346 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:24:22.341+0000 7efe29114700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 65461) UID: 1000 2021-03-09T09:24:22.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:24:31.347 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:24:31.348 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:24:31.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:24:31.348 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:24:31.349 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:24:31.349 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:24:31.352 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:24:31.352 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-03-09T09:24:38.332 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.27961275086055193) > weight (0.25) 2021-03-09T09:24:38.333 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6260151293867674) > weight (0.25) 2021-03-09T09:24:38.333 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7370499065327841) > weight (0.25) 2021-03-09T09:24:38.333 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8993405899855061) > weight (0.25) 2021-03-09T09:24:38.333 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T09:25:37.501 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T09:25:37.496+0000 7f082a074700 -1 reset not still connected to 0x55a990d33040 2021-03-09T09:25:37.502 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T09:25:37.496+0000 7f082a074700 -1 reset not still connected to 0x55a998838b60 2021-03-09T09:25:44.335 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5683198112178943) > weight (0.25) 2021-03-09T09:25:44.335 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.735582534686379) > weight (0.25) 2021-03-09T09:25:44.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8655624138363005) > weight (0.25) 2021-03-09T09:25:44.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5139370473982966) > weight (0.25) 2021-03-09T09:25:44.336 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T09:26:09.353 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.26176666676710514) > weight (0.25) 2021-03-09T09:26:09.376 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2714793736410013) > weight (0.25) 2021-03-09T09:26:09.377 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5998678931665958) > weight (0.25) 2021-03-09T09:26:09.377 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T09:26:09.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T09:26:09.378 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T09:26:09.378 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T09:26:09.379 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:26:09.379 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:26:09.375+0000 7f45fe977700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 64685) UID: 1000 2021-03-09T09:26:09.579 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:26:10.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:26:10.519+0000 7fc0262d2700 -1 rbd::mirror::ImageReplayer: 0x5630646f7200 [3/08a00e86-016b-46cb-bca2-dcf2f528e189] start: already running 2021-03-09T09:26:10.526 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:26:10.519+0000 7fc0262d2700 -1 rbd::mirror::ImageReplayer: 0x563068909900 [3/110cc5bb-34a2-4449-8efe-87f25e6218e2] start: already running 2021-03-09T09:26:10.527 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:26:10.519+0000 7f70d64bd700 -1 rbd::mirror::ImageReplayer: 0x562b3e257400 [3/3b232a3b-57f6-42ac-8d1f-68890a58c5ed] start: already running 2021-03-09T09:26:10.527 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T09:26:10.523+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad5106b10 2021-03-09T09:26:10.530 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T09:26:10.527+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9b95040 2021-03-09T09:26:10.534 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T09:26:10.527+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9b96ff0 2021-03-09T09:26:15.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:26:15.481 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T09:26:15.481 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T09:26:15.481 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T09:26:15.484 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T09:26:15.484 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T09:26:57.337 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.888375931996067) > weight (0.25) 2021-03-09T09:26:57.339 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.29166737553097455) > weight (0.25) 2021-03-09T09:26:57.339 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.43423938782550675) > weight (0.25) 2021-03-09T09:26:57.340 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.952201906509977) > weight (0.25) 2021-03-09T09:26:57.341 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T09:27:24.485 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8905367852895067) > weight (0.25) 2021-03-09T09:27:24.485 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9597027720050385) > weight (0.25) 2021-03-09T09:27:24.486 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T09:27:24.486 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T09:27:24.487 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T09:27:24.487 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T09:27:24.482+0000 7fd4e20ac700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 63816) UID: 1000 2021-03-09T09:27:24.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:27:27.487 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T09:27:27.488 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:27:27.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:27:27.489 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T09:27:27.489 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T09:27:27.489 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T09:27:27.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T09:27:27.492 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T09:28:33.493 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5122804826457811) > weight (0.25) 2021-03-09T09:28:33.526 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6077354229485987) > weight (0.25) 2021-03-09T09:28:33.526 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8190776690483499) > weight (0.25) 2021-03-09T09:28:33.526 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8836173994025479) > weight (0.25) 2021-03-09T09:28:33.527 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-03-09T09:28:37.342 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.25507487690418296) > weight (0.25) 2021-03-09T09:28:37.343 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.38183162213730215) > weight (0.25) 2021-03-09T09:28:37.343 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9903641820403095) > weight (0.25) 2021-03-09T09:28:37.343 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.34876405759330364) > weight (0.25) 2021-03-09T09:28:37.344 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-03-09T09:29:48.528 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8029134699197681) > weight (0.25) 2021-03-09T09:29:48.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.91143008485627) > weight (0.25) 2021-03-09T09:29:48.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9574985646902846) > weight (0.25) 2021-03-09T09:29:48.531 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8401892231728281) > weight (0.25) 2021-03-09T09:29:48.532 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T09:30:03.345 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7091023313005039) > weight (0.25) 2021-03-09T09:30:03.348 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3500488539547594) > weight (0.25) 2021-03-09T09:30:03.348 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3111036125548765) > weight (0.25) 2021-03-09T09:30:03.349 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.34892966250075597) > weight (0.25) 2021-03-09T09:30:03.349 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T09:30:48.533 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:30:48.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:30:48.564 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T09:30:48.565 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:30:48.558+0000 7fc0272d4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 65651) UID: 1000 2021-03-09T09:30:48.765 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:30:56.565 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:30:56.566 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:30:56.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:30:56.567 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:30:56.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:30:56.567 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:30:56.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:30:56.571 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T09:31:37.350 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T09:31:37.351 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T09:31:37.352 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T09:31:37.352 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T09:31:37.346+0000 7fbcd7246700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 65560) UID: 1000 2021-03-09T09:31:37.553 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:31:41.352 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T09:31:41.353 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:31:41.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:31:41.354 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T09:31:41.354 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T09:31:41.355 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T09:31:41.357 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T09:31:41.358 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T09:32:19.572 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6169123355780847) > weight (0.25) 2021-03-09T09:32:19.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8751099444421947) > weight (0.25) 2021-03-09T09:32:19.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.35338778703877927) > weight (0.25) 2021-03-09T09:32:19.573 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9004268180175724) > weight (0.25) 2021-03-09T09:32:19.573 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-03-09T09:33:06.359 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8240992449865705) > weight (0.25) 2021-03-09T09:33:06.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6112055500629777) > weight (0.25) 2021-03-09T09:33:06.363 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.987176342010369) > weight (0.25) 2021-03-09T09:33:06.363 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6066549068281755) > weight (0.25) 2021-03-09T09:33:06.363 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T09:33:55.574 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.29266729606346664) > weight (0.25) 2021-03-09T09:33:55.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4962070735109143) > weight (0.25) 2021-03-09T09:33:55.576 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5928962091470185) > weight (0.25) 2021-03-09T09:33:55.577 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T09:33:55.578 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T09:33:55.579 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T09:33:55.580 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:33:55.575+0000 7efefe12b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 65754) UID: 1000 2021-03-09T09:33:55.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:33:58.580 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T09:33:58.581 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:33:58.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:33:58.582 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T09:33:58.583 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T09:33:58.584 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T09:33:58.587 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T09:33:58.587 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T09:34:10.213 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T09:34:10.207+0000 7fc0e0d45700 -1 reset not still connected to 0x55e93d572a90 2021-03-09T09:34:10.218 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T09:34:10.211+0000 7fc0e0d45700 -1 reset not still connected to 0x55e93d574630 2021-03-09T09:34:10.218 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T09:34:10.211+0000 7fc0e0d45700 -1 reset not still connected to 0x55e93d575740 2021-03-09T09:34:55.364 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T09:34:55.367 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T09:34:55.368 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T09:34:55.369 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T09:34:55.362+0000 7f57de700700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 66057) UID: 1000 2021-03-09T09:34:55.569 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:35:03.369 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T09:35:03.370 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:35:03.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:35:03.371 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T09:35:03.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T09:35:03.372 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T09:35:03.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T09:35:03.375 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T09:35:24.107 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T09:35:24.098+0000 7f082a074700 -1 reset not still connected to 0x55a9a381a3c0 2021-03-09T09:35:38.588 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7545282570847925) > weight (0.25) 2021-03-09T09:35:38.591 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T09:35:38.592 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T09:35:38.592 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T09:35:38.593 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:35:38.585+0000 7f70d74bf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 65269) UID: 1000 2021-03-09T09:35:38.994 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:35:42.593 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T09:35:42.601 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:35:42.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:35:42.602 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T09:35:42.602 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T09:35:42.603 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T09:35:42.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T09:35:42.607 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T09:36:47.377 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T09:36:47.381 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T09:36:47.382 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T09:36:47.386 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T09:36:47.380+0000 7f0c11eaa700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 66251) UID: 1000 2021-03-09T09:36:47.587 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:36:54.383 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T09:36:54.384 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:36:54.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:36:54.385 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T09:36:54.385 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T09:36:54.385 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T09:36:54.388 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T09:36:54.388 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T09:37:31.607 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:37:31.610 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:37:31.611 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T09:37:31.612 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:37:31.607+0000 7fd92da14700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 65957) UID: 1000 2021-03-09T09:37:31.812 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:37:40.612 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:37:40.614 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:37:40.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:37:40.614 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:37:40.615 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:37:40.617 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:37:40.628 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:37:40.629 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T09:38:27.390 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T09:38:27.394 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T09:38:27.394 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T09:38:27.395 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T09:38:27.391+0000 7f944dfb2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 66443) UID: 1000 2021-03-09T09:38:27.596 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:38:30.396 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T09:38:30.396 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:38:30.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:38:30.397 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T09:38:30.397 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T09:38:30.398 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T09:38:30.404 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T09:38:30.405 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T09:38:52.630 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9134730591460979) > weight (0.25) 2021-03-09T09:38:52.632 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4989990987851136) > weight (0.25) 2021-03-09T09:38:52.632 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4291878344014234) > weight (0.25) 2021-03-09T09:38:52.632 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T09:38:52.633 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T09:38:52.633 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T09:38:52.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:38:52.630+0000 7f8f8bc05700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 66149) UID: 1000 2021-03-09T09:38:52.835 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:39:01.634 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T09:39:01.635 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:39:01.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:39:01.636 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T09:39:01.636 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T09:39:01.636 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T09:39:01.639 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T09:39:01.640 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T09:39:40.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9260967213913661) > weight (0.25) 2021-03-09T09:39:40.408 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T09:39:40.409 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T09:39:40.410 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T09:39:40.411 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T09:39:40.405+0000 7fc6cd3af700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 64102) UID: 1000 2021-03-09T09:39:40.611 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:39:41.411 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T09:39:41.412 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:39:41.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:39:41.412 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T09:39:41.413 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T09:39:41.413 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T09:39:41.416 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T09:39:41.416 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T09:40:38.641 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:40:38.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:40:38.647 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T09:40:38.647 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:40:38.644+0000 7feab67e4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 66535) UID: 1000 2021-03-09T09:40:38.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:40:41.647 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:40:41.648 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:40:41.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:40:41.649 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:40:41.649 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:40:41.649 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:40:41.652 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:40:41.652 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T09:40:42.417 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4238242985904698) > weight (0.25) 2021-03-09T09:40:42.418 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6758753203033694) > weight (0.25) 2021-03-09T09:40:42.418 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2580927285174891) > weight (0.25) 2021-03-09T09:40:42.418 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.858155918416448) > weight (0.25) 2021-03-09T09:40:42.418 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T09:42:04.419 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6820530027957568) > weight (0.25) 2021-03-09T09:42:04.423 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3012576282328766) > weight (0.25) 2021-03-09T09:42:04.423 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.37558288741953993) > weight (0.25) 2021-03-09T09:42:04.423 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2512979647690766) > weight (0.25) 2021-03-09T09:42:04.424 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T09:42:14.653 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:42:14.654 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:42:14.655 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T09:42:14.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:42:14.651+0000 7f1efdeb4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 66918) UID: 1000 2021-03-09T09:42:14.856 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:42:15.656 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:42:15.656 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:42:15.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:42:15.657 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:42:15.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:42:15.658 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:42:15.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:42:15.662 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T09:43:54.425 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T09:43:54.429 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T09:43:54.430 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T09:43:54.432 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T09:43:54.425+0000 7f5ae49ff700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 66634) UID: 1000 2021-03-09T09:43:54.632 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:44:01.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9437042070302228) > weight (0.25) 2021-03-09T09:44:01.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.44226690925021317) > weight (0.25) 2021-03-09T09:44:01.663 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T09:44:01.664 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T09:44:01.664 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T09:44:01.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T09:44:01.661+0000 7f4e17bb1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 65854) UID: 1000 2021-03-09T09:44:02.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:44:02.431 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T09:44:02.431 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:44:02.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:44:02.432 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T09:44:02.433 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T09:44:02.433 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T09:44:02.436 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T09:44:02.436 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T09:44:02.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:44:02.833+0000 7f1d1b9ef700 -1 rbd::mirror::ImageReplayer: 0x5568dacf5200 [3/08a00e86-016b-46cb-bca2-dcf2f528e189] start: already running 2021-03-09T09:44:02.840 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:44:02.833+0000 7f1d1b9ef700 -1 rbd::mirror::ImageReplayer: 0x5568def64000 [3/110cc5bb-34a2-4449-8efe-87f25e6218e2] start: already running 2021-03-09T09:44:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:44:02.833+0000 7f7dc7fd0700 -1 rbd::mirror::ImageReplayer: 0x5625327fb400 [3/7c5811a1-6f19-4292-9a3a-335262621df8] start: already running 2021-03-09T09:44:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:44:02.833+0000 7f7dc77cf700 -1 rbd::mirror::ImageReplayer: 0x562536d5c000 [3/82811dfa-9edb-4d58-845c-054f2ae371f8] start: already running 2021-03-09T09:44:02.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:44:02.833+0000 7f7dc7fd0700 -1 rbd::mirror::ImageReplayer: 0x5625327fcf80 [3/aec0a6af-f0ca-401c-a5d1-8f250674262a] start: already running 2021-03-09T09:44:02.843 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:44:02.837+0000 7f7dc77cf700 -1 rbd::mirror::ImageReplayer: 0x5625327fcd00 [3/bd4ef0c3-a3a7-42f2-8034-c828cab83717] start: already running 2021-03-09T09:44:02.845 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:44:02.837+0000 7f7dc77cf700 -1 rbd::mirror::ImageReplayer: 0x5625327fd200 [3/c6587b12-8826-47cf-8608-1a4b4fd5600a] start: already running 2021-03-09T09:44:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:44:02.837+0000 7f1d1b9ef700 -1 rbd::mirror::ImageReplayer: 0x5568dacf2f00 [3/1957e71d-23c8-464e-a815-0768c7c29be6] start: already running 2021-03-09T09:44:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:44:02.837+0000 7f1d1b9ef700 -1 rbd::mirror::ImageReplayer: 0x5568dacf3180 [3/36445824-fddb-4928-a242-dc49edee6cfa] start: already running 2021-03-09T09:44:02.846 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:44:02.837+0000 7f1d1b9ef700 -1 rbd::mirror::ImageReplayer: 0x5568dacf3400 [3/5f5c4aa6-1fcb-40a3-83e8-e1123b50fd68] start: already running 2021-03-09T09:44:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:44:02.837+0000 7f1d1b1ee700 -1 rbd::mirror::ImageReplayer: 0x5568dacf4f80 [3/76a412aa-913f-4f54-ad5f-0dbad3c607cd] start: already running 2021-03-09T09:44:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:44:02.837+0000 7f1d1b1ee700 -1 rbd::mirror::ImageReplayer: 0x5568def64280 [3/941e8ddf-64f0-4f25-89c8-9cfbbec2992f] start: already running 2021-03-09T09:44:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:44:02.837+0000 7f1d1b1ee700 -1 rbd::mirror::ImageReplayer: 0x5568dacf3680 [3/cbb866ff-7e4b-4de9-80d6-4f875228d852] start: already running 2021-03-09T09:44:02.847 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:44:02.837+0000 7f1d1b1ee700 -1 rbd::mirror::ImageReplayer: 0x5568dacf3900 [3/dc036c7f-f3e2-4032-b379-9d4600fb05ba] start: already running 2021-03-09T09:44:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:44:02.837+0000 7f1d1b1ee700 -1 rbd::mirror::ImageReplayer: 0x5568dacf3b80 [3/ee4a235d-1c78-45fb-8f1f-72d9026ab168] start: already running 2021-03-09T09:44:02.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:44:02.837+0000 7f1d1b1ee700 -1 rbd::mirror::ImageReplayer: 0x5568dacf5980 [3/fcee6884-fbbf-4b09-bfba-340ddeb5e81b] start: already running 2021-03-09T09:44:07.665 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T09:44:07.666 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:44:07.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:44:07.667 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T09:44:07.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T09:44:07.667 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T09:44:07.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T09:44:07.670 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T09:44:55.932 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:44:55.926+0000 7f901a51c700 -1 reset not still connected to 0x5588f96f08f0 2021-03-09T09:45:31.672 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:45:31.673 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:45:31.674 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T09:45:31.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:45:31.671+0000 7f1d1c1f0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 67018) UID: 1000 2021-03-09T09:45:32.075 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:45:40.675 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:45:40.675 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:45:40.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:45:40.676 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:45:40.676 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:45:40.677 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:45:40.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:45:40.686 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T09:45:50.437 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6849182963085967) > weight (0.25) 2021-03-09T09:45:50.438 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9473574889693606) > weight (0.25) 2021-03-09T09:45:50.438 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.43070263924730745) > weight (0.25) 2021-03-09T09:45:50.439 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7889444568251381) > weight (0.25) 2021-03-09T09:45:50.439 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T09:47:27.440 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.45284823217004644) > weight (0.25) 2021-03-09T09:47:27.442 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T09:47:27.443 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T09:47:27.444 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T09:47:27.445 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T09:47:27.441+0000 7f1b1d963700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 66825) UID: 1000 2021-03-09T09:47:27.645 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:47:30.445 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T09:47:30.445 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:47:30.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:47:30.446 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T09:47:30.447 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T09:47:30.447 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T09:47:30.450 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T09:47:30.451 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T09:47:31.687 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8710795888632032) > weight (0.25) 2021-03-09T09:47:31.687 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.271852094191788) > weight (0.25) 2021-03-09T09:47:31.688 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6801516593205434) > weight (0.25) 2021-03-09T09:47:31.688 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6351741331963142) > weight (0.25) 2021-03-09T09:47:31.688 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T09:48:48.689 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6801539265563029) > weight (0.25) 2021-03-09T09:48:48.731 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7022011452076817) > weight (0.25) 2021-03-09T09:48:48.732 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.37412705729745854) > weight (0.25) 2021-03-09T09:48:48.732 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T09:48:48.733 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T09:48:48.734 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T09:48:48.735 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:48:48.728+0000 7f39fa1d3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 66725) UID: 1000 2021-03-09T09:48:48.935 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:48:50.736 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T09:48:50.736 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:48:50.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:48:50.737 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T09:48:50.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T09:48:50.738 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T09:48:50.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T09:48:50.741 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T09:49:11.452 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7057487248478503) > weight (0.25) 2021-03-09T09:49:11.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8578271676588115) > weight (0.25) 2021-03-09T09:49:11.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.2566772929252784) > weight (0.25) 2021-03-09T09:49:11.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.25365594506716904) > weight (0.25) 2021-03-09T09:49:11.456 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T09:50:14.742 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:50:14.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:50:14.748 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T09:50:14.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:50:14.742+0000 7fcfd3806700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 67378) UID: 1000 2021-03-09T09:50:14.953 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:50:22.749 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:50:22.750 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:50:22.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:50:22.750 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:50:22.751 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:50:22.751 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:50:22.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:50:22.754 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T09:51:06.457 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T09:51:06.461 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T09:51:06.461 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T09:51:06.462 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T09:51:06.457+0000 7f1919d30700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 67184) UID: 1000 2021-03-09T09:51:06.662 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:51:08.462 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T09:51:08.463 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:51:08.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:51:08.464 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T09:51:08.464 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T09:51:08.465 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T09:51:08.467 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T09:51:08.468 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T09:51:38.755 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.36217842727236516) > weight (0.25) 2021-03-09T09:51:38.756 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T09:51:38.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T09:51:38.757 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T09:51:38.758 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:51:38.749+0000 7f7dc87d1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 66343) UID: 1000 2021-03-09T09:51:38.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:51:40.758 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T09:51:40.759 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:51:40.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:51:40.760 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T09:51:40.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T09:51:40.760 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T09:51:40.763 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T09:51:40.763 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T09:52:31.469 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9897847967982144) > weight (0.25) 2021-03-09T09:52:31.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9735793691916562) > weight (0.25) 2021-03-09T09:52:31.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4586893602462845) > weight (0.25) 2021-03-09T09:52:31.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5036940402540301) > weight (0.25) 2021-03-09T09:52:31.474 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-03-09T09:52:32.176 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:52:32.169+0000 7f901a51c700 -1 reset not still connected to 0x5588e7f43040 2021-03-09T09:52:32.176 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:52:32.169+0000 7f901a51c700 -1 reset not still connected to 0x5588fb0bb5f0 2021-03-09T09:52:32.177 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:52:32.169+0000 7f901a51c700 -1 reset not still connected to 0x5588fdfc1930 2021-03-09T09:52:56.764 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.42250308764241007) > weight (0.25) 2021-03-09T09:52:56.765 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6813463505697092) > weight (0.25) 2021-03-09T09:52:56.765 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.920021939267495) > weight (0.25) 2021-03-09T09:52:56.765 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6261048764945685) > weight (0.25) 2021-03-09T09:52:56.766 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T09:53:49.474 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8775703390784113) > weight (0.25) 2021-03-09T09:53:49.476 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5712668295948714) > weight (0.25) 2021-03-09T09:53:49.477 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T09:53:49.477 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T09:53:49.478 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T09:53:49.479 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T09:53:49.475+0000 7f3c27ac2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 64495) UID: 1000 2021-03-09T09:53:49.679 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:53:58.479 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T09:53:58.479 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:53:58.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:53:58.480 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T09:53:58.481 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T09:53:58.481 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T09:53:58.484 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T09:53:58.484 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T09:54:22.704 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T09:54:22.698+0000 7f082a074700 -1 reset not still connected to 0x55a9a25c2ff0 2021-03-09T09:54:22.704 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T09:54:22.698+0000 7f082a074700 -1 reset not still connected to 0x55a9a25c30c0 2021-03-09T09:54:22.723 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T09:54:22.718+0000 7f082a074700 -1 reset not still connected to 0x55a9c4e719b0 2021-03-09T09:54:45.766 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T09:54:45.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T09:54:45.769 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T09:54:45.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:54:45.762+0000 7f576a0a4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 67672) UID: 1000 2021-03-09T09:54:45.970 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:54:48.770 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T09:54:48.770 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:54:48.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:54:48.771 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T09:54:48.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T09:54:48.772 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T09:54:48.775 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T09:54:48.775 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T09:55:37.847 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:55:37.839+0000 7f901a51c700 -1 reset not still connected to 0x5588fa399860 2021-03-09T09:55:37.850 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:55:37.843+0000 7f901a51c700 -1 reset not still connected to 0x5588fb0bb5f0 2021-03-09T09:55:37.851 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:55:37.843+0000 7f901a51c700 -1 reset not still connected to 0x5588fb0bc3c0 2021-03-09T09:55:37.854 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:55:37.847+0000 7f901a51c700 -1 reset not still connected to 0x5588fdfc16c0 2021-03-09T09:55:37.854 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:55:37.847+0000 7f901a51c700 -1 reset not still connected to 0x5588fdfc1930 2021-03-09T09:55:37.854 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:55:37.847+0000 7f901a51c700 -1 reset not still connected to 0x558901883860 2021-03-09T09:55:37.854 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:55:37.847+0000 7f901a51c700 -1 reset not still connected to 0x558901883ba0 2021-03-09T09:55:53.486 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46680414129152215) > weight (0.25) 2021-03-09T09:55:53.488 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T09:55:53.489 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T09:55:53.489 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T09:55:53.490 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T09:55:53.481+0000 7f0637080700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 67479) UID: 1000 2021-03-09T09:55:53.691 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:56:00.490 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T09:56:00.491 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:56:00.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:56:00.492 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T09:56:00.492 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T09:56:00.492 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T09:56:00.495 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T09:56:00.495 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T09:56:12.777 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7832979150801821) > weight (0.25) 2021-03-09T09:56:12.777 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.26773484177005746) > weight (0.25) 2021-03-09T09:56:12.777 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.36659022625451665) > weight (0.25) 2021-03-09T09:56:12.778 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T09:56:12.778 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T09:56:12.779 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T09:56:12.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:56:12.776+0000 7fa6184a1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 67572) UID: 1000 2021-03-09T09:56:12.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:56:21.780 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T09:56:21.782 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:56:21.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:56:21.783 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T09:56:21.784 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T09:56:21.784 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T09:56:21.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T09:56:21.787 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T09:57:43.497 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.49474573271248523) > weight (0.25) 2021-03-09T09:57:43.500 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2809546283118469) > weight (0.25) 2021-03-09T09:57:43.500 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8512698142102777) > weight (0.25) 2021-03-09T09:57:43.500 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6535407131149464) > weight (0.25) 2021-03-09T09:57:43.500 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T09:58:06.789 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8912961027481365) > weight (0.25) 2021-03-09T09:58:06.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9265175606359741) > weight (0.25) 2021-03-09T09:58:06.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8798676704186082) > weight (0.25) 2021-03-09T09:58:06.793 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.39197912359130394) > weight (0.25) 2021-03-09T09:58:06.793 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T09:59:27.501 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T09:59:27.506 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T09:59:27.506 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T09:59:27.507 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T09:59:27.501+0000 7fe76550b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 67772) UID: 1000 2021-03-09T09:59:27.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:59:32.508 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T09:59:32.508 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:59:32.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:59:32.509 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T09:59:32.510 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T09:59:32.510 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T09:59:32.513 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T09:59:32.514 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T09:59:40.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9288138334203495) > weight (0.25) 2021-03-09T09:59:40.795 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5179718180533398) > weight (0.25) 2021-03-09T09:59:40.795 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T09:59:40.795 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T09:59:40.796 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T09:59:40.796 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T09:59:40.796 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T09:59:40.797 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T09:59:40.789+0000 7f4b9fefe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 67275) UID: 1000 2021-03-09T09:59:41.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T09:59:41.979 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T09:59:41.972+0000 7f901a51c700 -1 reset not still connected to 0x558904ab1ba0 2021-03-09T09:59:41.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:59:41.973+0000 7f8efa112700 -1 rbd::mirror::ImageReplayer: 0x55e7a2f61b80 [3/0acd7cf2-0841-4351-804c-3096c0145f8f] start: already running 2021-03-09T09:59:41.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:59:41.973+0000 7f8efa112700 -1 rbd::mirror::ImageReplayer: 0x55e7a2f60f00 [3/28f924e2-e881-4421-bb68-c0bf69c20856] start: already running 2021-03-09T09:59:41.982 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:59:41.973+0000 7f8efa112700 -1 rbd::mirror::ImageReplayer: 0x55e7a2f61180 [3/35b68617-078e-413b-9699-cf8f31934816] start: already running 2021-03-09T09:59:41.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:59:41.977+0000 7f8ef9911700 -1 rbd::mirror::ImageReplayer: 0x55e7a2f62300 [3/08a00e86-016b-46cb-bca2-dcf2f528e189] start: already running 2021-03-09T09:59:41.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:59:41.977+0000 7f8efa112700 -1 rbd::mirror::ImageReplayer: 0x55e7a2f61b80 [3/0acd7cf2-0841-4351-804c-3096c0145f8f] start: already running 2021-03-09T09:59:41.984 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:59:41.977+0000 7f8ef9911700 -1 rbd::mirror::ImageReplayer: 0x55e7a2f60f00 [3/28f924e2-e881-4421-bb68-c0bf69c20856] start: already running 2021-03-09T09:59:41.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:59:41.977+0000 7f83afd24700 -1 rbd::mirror::ImageReplayer: 0x55e76dde7e00 [3/3b232a3b-57f6-42ac-8d1f-68890a58c5ed] start: already running 2021-03-09T09:59:41.987 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T09:59:41.980+0000 7fa08ad3c700 -1 reset not still connected to 0x564ace4034d0 2021-03-09T09:59:41.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:59:41.981+0000 7f6f0b9a7700 -1 rbd::mirror::ImageReplayer: 0x55f3f3dd4000 [3/90629254-6fb9-41d9-b690-e94fb222ffa6] start: already running 2021-03-09T09:59:41.988 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:59:41.981+0000 7f6f0b9a7700 -1 rbd::mirror::ImageReplayer: 0x55f3efd93180 [3/9457c621-e2d5-43c5-bb20-fb14f55cf969] start: already running 2021-03-09T09:59:41.989 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T09:59:41.981+0000 7f6f0b9a7700 -1 rbd::mirror::ImageReplayer: 0x55f3efd95c00 [3/cbb866ff-7e4b-4de9-80d6-4f875228d852] start: already running 2021-03-09T09:59:41.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:59:41.989+0000 7f83afd24700 -1 rbd::mirror::ImageReplayer: 0x55e76dde6c80 [3/18a549b3-61f2-46e5-af4d-1a9b02c6e0ad] start: already running 2021-03-09T09:59:41.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:59:41.989+0000 7f83af523700 -1 rbd::mirror::ImageReplayer: 0x55e76dde7e00 [3/3b232a3b-57f6-42ac-8d1f-68890a58c5ed] start: already running 2021-03-09T09:59:41.996 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:59:41.989+0000 7f83afd24700 -1 rbd::mirror::ImageReplayer: 0x55e76dde8800 [3/59893ab7-fc80-4b4a-85c5-aa491edb65bc] start: already running 2021-03-09T09:59:41.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:59:41.989+0000 7f83af523700 -1 rbd::mirror::ImageReplayer: 0x55e76dde9700 [3/6eb851a0-fa35-4de2-b7b1-277c2544ef7a] start: already running 2021-03-09T09:59:41.997 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T09:59:41.989+0000 7f83afd24700 -1 rbd::mirror::ImageReplayer: 0x55e76dde8d00 [3/82811dfa-9edb-4d58-845c-054f2ae371f8] start: already running 2021-03-09T09:59:42.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:59:42.001+0000 7f8efa112700 -1 rbd::mirror::ImageReplayer: 0x55e7a2f62300 [3/08a00e86-016b-46cb-bca2-dcf2f528e189] start: already running 2021-03-09T09:59:42.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:59:42.001+0000 7f8efa112700 -1 rbd::mirror::ImageReplayer: 0x55e7a2f61b80 [3/0acd7cf2-0841-4351-804c-3096c0145f8f] start: already running 2021-03-09T09:59:42.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:59:42.001+0000 7f8efa112700 -1 rbd::mirror::ImageReplayer: 0x55e7a2f60f00 [3/28f924e2-e881-4421-bb68-c0bf69c20856] start: already running 2021-03-09T09:59:42.009 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:59:42.001+0000 7f8efa112700 -1 rbd::mirror::ImageReplayer: 0x55e7a2f61180 [3/35b68617-078e-413b-9699-cf8f31934816] start: already running 2021-03-09T09:59:42.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T09:59:42.001+0000 7f8ef9911700 -1 rbd::mirror::ImageReplayer: 0x55e7a2f61680 [3/36445824-fddb-4928-a242-dc49edee6cfa] start: already running 2021-03-09T09:59:46.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T09:59:46.898 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T09:59:46.898 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T09:59:46.899 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T09:59:46.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T09:59:46.902 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T10:01:12.515 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2858015168247946) > weight (0.25) 2021-03-09T10:01:12.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.48358967525646956) > weight (0.25) 2021-03-09T10:01:12.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.83633761269661) > weight (0.25) 2021-03-09T10:01:12.522 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.37074899553950713) > weight (0.25) 2021-03-09T10:01:12.522 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T10:01:23.903 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6696610498106392) > weight (0.25) 2021-03-09T10:01:23.910 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.34243784026131063) > weight (0.25) 2021-03-09T10:01:23.910 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T10:01:23.911 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T10:01:23.914 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T10:01:23.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T10:01:23.907+0000 7f940edf8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 68441) UID: 1000 2021-03-09T10:01:24.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:01:27.915 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T10:01:27.915 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:01:27.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:01:27.916 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T10:01:27.917 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T10:01:27.917 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T10:01:27.920 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T10:01:27.921 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T10:02:27.922 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3629093284994067) > weight (0.25) 2021-03-09T10:02:27.925 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T10:02:27.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T10:02:27.926 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T10:02:27.927 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T10:02:27.922+0000 7f6f0c1a8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 67863) UID: 1000 2021-03-09T10:02:28.128 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:02:35.927 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T10:02:35.929 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:02:35.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:02:35.930 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T10:02:35.930 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T10:02:35.931 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T10:02:35.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T10:02:35.934 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T10:02:45.523 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7880416591448265) > weight (0.25) 2021-03-09T10:02:45.526 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T10:02:45.527 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T10:02:45.527 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T10:02:45.528 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T10:02:45.521+0000 7fe3c8864700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 68159) UID: 1000 2021-03-09T10:02:45.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:02:48.528 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T10:02:48.529 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:02:48.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:02:48.530 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T10:02:48.530 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T10:02:48.530 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T10:02:48.537 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T10:02:48.537 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T10:04:13.538 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T10:04:13.542 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T10:04:13.543 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T10:04:13.544 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T10:04:13.540+0000 7f92c86c6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 68350) UID: 1000 2021-03-09T10:04:13.744 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:04:17.544 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T10:04:17.544 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:04:17.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:04:17.545 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T10:04:17.546 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T10:04:17.546 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T10:04:17.549 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T10:04:17.549 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-03-09T10:04:26.935 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T10:04:26.940 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T10:04:26.940 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T10:04:26.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T10:04:26.936+0000 7f8efa913700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 68055) UID: 1000 2021-03-09T10:04:27.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:04:27.941 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T10:04:27.942 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:04:27.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:04:27.943 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T10:04:27.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T10:04:27.943 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T10:04:27.946 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T10:04:27.946 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T10:05:43.550 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6827634848951832) > weight (0.25) 2021-03-09T10:05:43.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9853577833959632) > weight (0.25) 2021-03-09T10:05:43.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7652430299334982) > weight (0.25) 2021-03-09T10:05:43.556 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4125132405361438) > weight (0.25) 2021-03-09T10:05:43.556 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T10:06:13.948 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5451034982180784) > weight (0.25) 2021-03-09T10:06:13.948 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T10:06:13.949 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T10:06:13.950 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T10:06:13.951 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T10:06:13.946+0000 7f7e75b34700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 68641) UID: 1000 2021-03-09T10:06:14.151 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:06:16.951 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T10:06:16.951 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:06:16.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:06:16.952 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T10:06:16.952 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T10:06:16.953 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T10:06:16.956 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T10:06:16.957 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T10:06:52.558 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6842499935239286) > weight (0.25) 2021-03-09T10:06:52.561 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T10:06:52.562 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T10:06:52.562 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T10:06:52.563 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T10:06:52.557+0000 7f2684b6f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 68740) UID: 1000 2021-03-09T10:06:52.763 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:06:56.563 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T10:06:56.564 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:06:56.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:06:56.565 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T10:06:56.565 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T10:06:56.565 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T10:06:56.568 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T10:06:56.569 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T10:07:44.958 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32658032492863254) > weight (0.25) 2021-03-09T10:07:44.959 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7716294149789384) > weight (0.25) 2021-03-09T10:07:44.960 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4187200174511797) > weight (0.25) 2021-03-09T10:07:44.960 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T10:07:44.960 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T10:07:44.962 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T10:07:44.962 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T10:07:44.956+0000 7f83b0525700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 68250) UID: 1000 2021-03-09T10:07:45.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:07:51.963 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T10:07:51.963 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:07:51.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:07:51.964 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T10:07:51.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T10:07:51.965 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T10:07:51.968 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T10:07:51.968 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T10:08:51.569 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.428425638652574) > weight (0.25) 2021-03-09T10:08:51.573 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T10:08:51.574 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T10:08:51.574 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T10:08:51.575 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T10:08:51.571+0000 7f1bf24eb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 69125) UID: 1000 2021-03-09T10:08:51.775 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:08:52.576 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T10:08:52.576 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:08:52.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:08:52.577 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T10:08:52.577 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T10:08:52.578 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T10:08:52.581 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T10:08:52.582 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T10:09:11.969 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.36020842450529866) > weight (0.25) 2021-03-09T10:09:11.970 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8741076584047428) > weight (0.25) 2021-03-09T10:09:11.970 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5349902639681179) > weight (0.25) 2021-03-09T10:09:11.971 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7465359614712918) > weight (0.25) 2021-03-09T10:09:11.971 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T10:10:25.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.645189329237424) > weight (0.25) 2021-03-09T10:10:25.976 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7434166395206994) > weight (0.25) 2021-03-09T10:10:25.977 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T10:10:25.978 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T10:10:25.978 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T10:10:25.978 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T10:10:25.979 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:10:25.980 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T10:10:25.974+0000 7fcc38c81700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 68542) UID: 1000 2021-03-09T10:10:26.179 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:10:32.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:10:32.081 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T10:10:32.081 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T10:10:32.082 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T10:10:32.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T10:10:32.085 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T10:10:40.582 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T10:10:40.583 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T10:10:40.584 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T10:10:40.584 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T10:10:40.577+0000 7f220d257700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 68833) UID: 1000 2021-03-09T10:10:40.785 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:10:47.584 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T10:10:47.585 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:10:47.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:10:47.586 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T10:10:47.586 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T10:10:47.586 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T10:10:47.589 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T10:10:47.589 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-03-09T10:12:08.590 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42035550955679735) > weight (0.25) 2021-03-09T10:12:08.594 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9394492020567595) > weight (0.25) 2021-03-09T10:12:08.594 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T10:12:08.595 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T10:12:08.596 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T10:12:08.596 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T10:12:08.596 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:12:08.597 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T10:12:08.592+0000 7f0c1d770700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 67964) UID: 1000 2021-03-09T10:12:08.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:12:14.086 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.26998101977445965) > weight (0.25) 2021-03-09T10:12:14.086 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4843280800471512) > weight (0.25) 2021-03-09T10:12:14.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8596491681897673) > weight (0.25) 2021-03-09T10:12:14.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9965778473693069) > weight (0.25) 2021-03-09T10:12:14.087 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T10:12:14.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:12:14.698 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T10:12:14.698 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T10:12:14.699 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T10:12:14.702 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T10:12:14.702 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T10:12:40.282 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:12:40.275+0000 7f082a074700 -1 reset not still connected to 0x55a99db96410 2021-03-09T10:12:40.282 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:12:40.275+0000 7f082a074700 -1 reset not still connected to 0x55a9a25c2ff0 2021-03-09T10:12:40.282 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:12:40.275+0000 7f082a074700 -1 reset not still connected to 0x55a9a25c30c0 2021-03-09T10:13:17.088 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T10:13:17.097 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T10:13:17.097 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T10:13:17.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T10:13:17.091+0000 7fbf4f836700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 68924) UID: 1000 2021-03-09T10:13:17.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:13:18.316 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T10:13:18.309+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad5b03040 2021-03-09T10:13:18.316 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T10:13:18.309+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9b94dd0 2021-03-09T10:13:18.316 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T10:13:18.309+0000 7fa08ad3c700 -1 reset not still connected to 0x564ae0e0ef20 2021-03-09T10:13:20.098 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T10:13:20.099 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:13:20.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:13:20.100 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T10:13:20.100 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T10:13:20.101 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T10:13:20.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T10:13:20.104 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T10:13:34.704 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9097191470881172) > weight (0.25) 2021-03-09T10:13:34.704 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.811950170549723) > weight (0.25) 2021-03-09T10:13:34.705 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5463234091543132) > weight (0.25) 2021-03-09T10:13:34.705 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T10:13:34.706 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T10:13:34.706 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T10:13:34.707 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T10:13:34.702+0000 7f4e4ffdf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 65369) UID: 1000 2021-03-09T10:13:34.907 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:13:35.878 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T10:13:35.870+0000 7faee9e8c700 -1 rbd::mirror::ImageReplayer: 0x560e12766f80 [3/18a549b3-61f2-46e5-af4d-1a9b02c6e0ad] start: already running 2021-03-09T10:13:38.707 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T10:13:38.707 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:13:38.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:13:38.708 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T10:13:38.708 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T10:13:38.708 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T10:13:38.711 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T10:13:38.711 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T10:14:43.105 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7724801227945995) > weight (0.25) 2021-03-09T10:14:43.107 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8046638857127218) > weight (0.25) 2021-03-09T10:14:43.108 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T10:14:43.108 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T10:14:43.109 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T10:14:43.113 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T10:14:43.105+0000 7f8498c52700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 69410) UID: 1000 2021-03-09T10:14:43.313 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:14:49.110 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T10:14:49.110 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:14:49.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:14:49.111 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T10:14:49.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T10:14:49.112 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T10:14:49.115 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T10:14:49.115 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T10:14:54.347 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:14:54.341+0000 7f082a074700 -1 reset not still connected to 0x55a99f7e55f0 2021-03-09T10:14:54.348 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:14:54.341+0000 7f082a074700 -1 reset not still connected to 0x55a9f121b5f0 2021-03-09T10:15:08.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.43223119578376035) > weight (0.25) 2021-03-09T10:15:08.713 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7073274307738585) > weight (0.25) 2021-03-09T10:15:08.713 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T10:15:08.714 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T10:15:08.714 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T10:15:08.715 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T10:15:08.709+0000 7f1c37b9a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 69600) UID: 1000 2021-03-09T10:15:08.916 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:15:14.715 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T10:15:14.715 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:15:14.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:15:14.717 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T10:15:14.717 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T10:15:14.717 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T10:15:14.724 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T10:15:14.724 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-03-09T10:15:16.425 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T10:15:16.421+0000 7f901a51c700 -1 reset not still connected to 0x558904ab0a90 2021-03-09T10:16:06.116 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9350032965314095) > weight (0.25) 2021-03-09T10:16:06.117 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5714333287042797) > weight (0.25) 2021-03-09T10:16:06.117 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.842486455641397) > weight (0.25) 2021-03-09T10:16:06.117 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8882011638349978) > weight (0.25) 2021-03-09T10:16:06.117 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T10:16:25.725 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5764735705328116) > weight (0.25) 2021-03-09T10:16:25.726 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7002858074411091) > weight (0.25) 2021-03-09T10:16:25.726 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T10:16:25.727 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T10:16:25.727 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T10:16:25.728 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T10:16:25.728 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:16:25.729 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T10:16:25.723+0000 7f7afa3e3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 69986) UID: 1000 2021-03-09T10:16:25.929 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:16:31.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:16:31.830 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T10:16:31.831 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T10:16:31.831 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T10:16:31.834 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T10:16:31.834 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T10:17:11.118 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8767940349660434) > weight (0.25) 2021-03-09T10:17:11.119 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T10:17:11.119 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T10:17:11.120 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T10:17:11.121 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T10:17:11.115+0000 7f7af7d5d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 69026) UID: 1000 2021-03-09T10:17:11.321 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:17:18.120 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T10:17:18.121 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:17:18.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:17:18.122 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T10:17:18.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T10:17:18.123 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T10:17:18.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T10:17:18.128 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T10:17:37.835 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.907077051032656) > weight (0.25) 2021-03-09T10:17:37.836 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7384615138651486) > weight (0.25) 2021-03-09T10:17:37.836 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5578001716177681) > weight (0.25) 2021-03-09T10:17:37.836 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5110683841348547) > weight (0.25) 2021-03-09T10:17:37.836 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T10:18:21.129 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3522684613153052) > weight (0.25) 2021-03-09T10:18:21.129 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4739380246649021) > weight (0.25) 2021-03-09T10:18:21.130 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8611173040453267) > weight (0.25) 2021-03-09T10:18:21.130 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6558223784779738) > weight (0.25) 2021-03-09T10:18:21.130 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T10:19:18.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9416701585992764) > weight (0.25) 2021-03-09T10:19:18.839 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T10:19:18.840 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T10:19:18.841 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T10:19:18.842 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T10:19:18.836+0000 7f024c0d0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 69317) UID: 1000 2021-03-09T10:19:19.042 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:19:22.841 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T10:19:22.842 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:19:22.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:19:22.843 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T10:19:22.843 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T10:19:22.844 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T10:19:22.847 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T10:19:22.847 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-03-09T10:19:38.934 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:19:38.928+0000 7f082a074700 -1 reset not still connected to 0x55a9938b6680 2021-03-09T10:19:38.937 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:19:38.932+0000 7f082a074700 -1 reset not still connected to 0x55a9a0a4a630 2021-03-09T10:20:11.131 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3814382151960308) > weight (0.25) 2021-03-09T10:20:11.133 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T10:20:11.133 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T10:20:11.135 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T10:20:11.135 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T10:20:11.132+0000 7f60de55d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 70171) UID: 1000 2021-03-09T10:20:11.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:20:20.136 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T10:20:20.136 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:20:20.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:20:20.137 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T10:20:20.137 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T10:20:20.138 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T10:20:20.143 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T10:20:20.143 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T10:20:58.848 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3105762671770337) > weight (0.25) 2021-03-09T10:20:58.848 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7011281317310617) > weight (0.25) 2021-03-09T10:20:58.848 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9189572911724896) > weight (0.25) 2021-03-09T10:20:58.849 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2866382124986271) > weight (0.25) 2021-03-09T10:20:58.849 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T10:22:03.145 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T10:22:03.146 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T10:22:03.147 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T10:22:03.148 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T10:22:03.142+0000 7f4e97b78700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 69692) UID: 1000 2021-03-09T10:22:03.348 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:22:07.148 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T10:22:07.148 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:22:07.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:22:07.149 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T10:22:07.149 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T10:22:07.149 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T10:22:07.152 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T10:22:07.153 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T10:22:25.850 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T10:22:25.851 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T10:22:25.851 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T10:22:25.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T10:22:25.853+0000 7faeeae8e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 69509) UID: 1000 2021-03-09T10:22:26.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:22:26.853 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T10:22:26.853 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:22:26.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:22:26.854 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T10:22:26.854 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T10:22:26.855 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T10:22:26.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T10:22:26.858 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T10:23:38.860 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7621748909483927) > weight (0.25) 2021-03-09T10:23:38.863 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8367773471127162) > weight (0.25) 2021-03-09T10:23:38.864 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8772939411350333) > weight (0.25) 2021-03-09T10:23:38.864 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.33351075370023675) > weight (0.25) 2021-03-09T10:23:38.864 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T10:23:49.154 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7305209044355265) > weight (0.25) 2021-03-09T10:23:49.154 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T10:23:49.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T10:23:49.156 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T10:23:49.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T10:23:49.152+0000 7fe189b37700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 70366) UID: 1000 2021-03-09T10:23:49.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:23:53.157 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T10:23:53.157 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:23:53.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:23:53.158 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T10:23:53.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T10:23:53.159 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T10:23:53.162 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T10:23:53.162 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T10:25:08.866 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7081702176279663) > weight (0.25) 2021-03-09T10:25:08.899 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2902098242720138) > weight (0.25) 2021-03-09T10:25:08.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6487738834766793) > weight (0.25) 2021-03-09T10:25:08.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.2804103382671894) > weight (0.25) 2021-03-09T10:25:08.900 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T10:25:18.163 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T10:25:18.164 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T10:25:18.164 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T10:25:18.165 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T10:25:18.162+0000 7f237449a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 70465) UID: 1000 2021-03-09T10:25:18.365 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:25:22.166 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T10:25:22.166 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:25:22.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:25:22.167 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T10:25:22.167 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T10:25:22.168 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T10:25:22.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T10:25:22.171 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-03-09T10:26:08.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2944598516465208) > weight (0.25) 2021-03-09T10:26:08.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4667426079712311) > weight (0.25) 2021-03-09T10:26:08.903 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7324654578244257) > weight (0.25) 2021-03-09T10:26:08.903 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T10:26:08.904 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T10:26:08.904 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T10:26:08.905 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T10:26:08.897+0000 7fad1579a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 69791) UID: 1000 2021-03-09T10:26:09.105 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:26:10.905 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T10:26:10.905 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:26:10.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:26:10.906 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T10:26:10.906 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T10:26:10.907 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T10:26:10.910 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T10:26:10.910 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-03-09T10:26:30.173 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.554375824342746) > weight (0.25) 2021-03-09T10:26:30.173 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8909317666747699) > weight (0.25) 2021-03-09T10:26:30.173 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8448961115446786) > weight (0.25) 2021-03-09T10:26:30.174 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8151488925064942) > weight (0.25) 2021-03-09T10:26:30.174 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T10:27:46.911 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T10:27:46.954 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T10:27:46.955 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T10:27:46.955 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T10:27:46.952+0000 7fd456abf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 70564) UID: 1000 2021-03-09T10:27:47.156 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:27:49.956 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T10:27:49.956 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:27:49.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:27:49.957 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T10:27:49.957 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T10:27:49.958 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T10:27:49.960 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T10:27:49.961 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T10:28:26.175 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7722839093498787) > weight (0.25) 2021-03-09T10:28:26.176 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3882134207480459) > weight (0.25) 2021-03-09T10:28:26.177 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5187204065367662) > weight (0.25) 2021-03-09T10:28:26.177 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8544378374912079) > weight (0.25) 2021-03-09T10:28:26.178 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T10:29:22.962 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7606545947218788) > weight (0.25) 2021-03-09T10:29:22.966 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.43177400516611375) > weight (0.25) 2021-03-09T10:29:22.966 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8847575958904755) > weight (0.25) 2021-03-09T10:29:22.966 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T10:29:22.968 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T10:29:22.968 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T10:29:22.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T10:29:22.962+0000 7f8a4fe8a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 70860) UID: 1000 2021-03-09T10:29:23.169 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:29:31.969 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T10:29:31.971 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:29:31.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:29:31.972 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T10:29:31.973 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T10:29:31.974 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T10:29:31.978 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T10:29:31.979 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T10:29:53.179 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4701839895763501) > weight (0.25) 2021-03-09T10:29:53.180 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T10:29:53.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T10:29:53.181 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T10:29:53.182 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T10:29:53.174+0000 7efe8c8af700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 70655) UID: 1000 2021-03-09T10:29:53.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:29:55.267 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T10:29:55.262+0000 7fde32d68700 -1 rbd::mirror::ImageReplayer: 0x564ed40b6f00 [3/bd4ef0c3-a3a7-42f2-8034-c828cab83717] start: already running 2021-03-09T10:29:58.182 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T10:29:58.182 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:29:58.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:29:58.183 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T10:29:58.183 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T10:29:58.184 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T10:29:58.186 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T10:29:58.187 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T10:30:48.980 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.36052077387370063) > weight (0.25) 2021-03-09T10:30:48.981 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9033301429674722) > weight (0.25) 2021-03-09T10:30:48.981 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T10:30:48.982 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T10:30:48.982 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T10:30:48.983 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T10:30:48.981+0000 7fde33569700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 70077) UID: 1000 2021-03-09T10:30:49.384 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:30:50.236 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:30:50.225+0000 7f082a074700 -1 reset not still connected to 0x55aa12b1eb60 2021-03-09T10:30:55.983 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T10:30:55.984 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:30:55.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:30:55.984 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T10:30:55.985 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T10:30:55.985 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T10:30:55.988 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T10:30:55.988 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T10:31:44.188 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.45819354067534923) > weight (0.25) 2021-03-09T10:31:44.189 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.48324557629988274) > weight (0.25) 2021-03-09T10:31:44.189 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5236197921726713) > weight (0.25) 2021-03-09T10:31:44.190 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6525192718203063) > weight (0.25) 2021-03-09T10:31:44.190 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-03-09T10:32:20.202 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:32:20.195+0000 7f082a074700 -1 reset not still connected to 0x55a990d32340 2021-03-09T10:32:20.203 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:32:20.195+0000 7f082a074700 -1 reset not still connected to 0x55a990d34ff0 2021-03-09T10:32:20.204 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:32:20.199+0000 7f082a074700 -1 reset not still connected to 0x55aa12b1eb60 2021-03-09T10:32:24.989 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4921289359479809) > weight (0.25) 2021-03-09T10:32:24.992 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7611485497249061) > weight (0.25) 2021-03-09T10:32:24.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3675681649677647) > weight (0.25) 2021-03-09T10:32:24.993 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9169377773927032) > weight (0.25) 2021-03-09T10:32:24.993 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T10:33:42.191 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.753836224677101) > weight (0.25) 2021-03-09T10:33:42.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.570073137853901) > weight (0.25) 2021-03-09T10:33:42.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8160932288852143) > weight (0.25) 2021-03-09T10:33:42.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7401351550467937) > weight (0.25) 2021-03-09T10:33:42.196 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T10:34:16.995 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.532217204423647) > weight (0.25) 2021-03-09T10:34:17.017 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9253636054244203) > weight (0.25) 2021-03-09T10:34:17.017 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.501581616280752) > weight (0.25) 2021-03-09T10:34:17.018 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3872990200327602) > weight (0.25) 2021-03-09T10:34:17.018 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T10:35:19.197 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7192256593757484) > weight (0.25) 2021-03-09T10:35:19.203 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T10:35:19.203 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T10:35:19.204 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T10:35:19.204 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T10:35:19.204 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:35:19.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T10:35:19.200+0000 7ff32ef61700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 71210) UID: 1000 2021-03-09T10:35:19.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:35:25.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:35:25.307 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T10:35:25.307 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T10:35:25.308 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T10:35:25.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T10:35:25.311 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T10:36:10.019 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7866491513674078) > weight (0.25) 2021-03-09T10:36:10.020 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4257246444895245) > weight (0.25) 2021-03-09T10:36:10.020 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3427737951451306) > weight (0.25) 2021-03-09T10:36:10.020 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.38658994900400434) > weight (0.25) 2021-03-09T10:36:10.020 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T10:37:18.312 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.27204655063875094) > weight (0.25) 2021-03-09T10:37:19.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.886774848154791) > weight (0.25) 2021-03-09T10:37:19.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6809227760495669) > weight (0.25) 2021-03-09T10:37:19.006 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7561075823358133) > weight (0.25) 2021-03-09T10:37:19.007 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T10:37:53.021 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6932724622032846) > weight (0.25) 2021-03-09T10:37:53.022 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T10:37:53.022 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T10:37:53.023 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T10:37:53.023 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T10:37:53.017+0000 7f127081e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 70271) UID: 1000 2021-03-09T10:37:53.224 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:38:01.023 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T10:38:01.024 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:38:01.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:38:01.025 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T10:38:01.025 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T10:38:01.025 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T10:38:01.028 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T10:38:01.028 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T10:38:02.194 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:38:02.185+0000 7f082a074700 -1 reset not still connected to 0x55a9f121a750 2021-03-09T10:38:02.194 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:38:02.185+0000 7f082a074700 -1 reset not still connected to 0x55a9f121cb10 2021-03-09T10:38:58.008 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T10:38:58.043 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T10:38:58.044 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T10:38:58.045 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T10:38:58.040+0000 7f9a8a440700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 70759) UID: 1000 2021-03-09T10:38:58.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:39:00.045 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T10:39:00.046 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:39:00.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:39:00.047 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T10:39:00.047 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T10:39:00.047 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T10:39:00.052 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T10:39:00.053 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T10:39:52.030 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5605590808377918) > weight (0.25) 2021-03-09T10:39:52.030 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6222139204322875) > weight (0.25) 2021-03-09T10:39:52.031 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T10:39:52.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T10:39:52.032 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T10:39:52.033 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T10:39:52.027+0000 7f1670ed2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 71311) UID: 1000 2021-03-09T10:39:52.233 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:39:56.032 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T10:39:56.033 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:39:56.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:39:56.034 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T10:39:56.034 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T10:39:56.035 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T10:39:56.037 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T10:39:56.038 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T10:40:41.054 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5435267779710009) > weight (0.25) 2021-03-09T10:40:41.055 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8772632356929779) > weight (0.25) 2021-03-09T10:40:41.055 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9396902507707734) > weight (0.25) 2021-03-09T10:40:41.055 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9377875677610313) > weight (0.25) 2021-03-09T10:40:41.055 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T10:41:05.039 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T10:41:05.040 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T10:41:05.040 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T10:41:05.041 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T10:41:05.034+0000 7fae44699700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 70952) UID: 1000 2021-03-09T10:41:05.242 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:41:07.041 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T10:41:07.042 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:41:07.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:41:07.042 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T10:41:07.043 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T10:41:07.043 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T10:41:07.046 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T10:41:07.046 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T10:42:22.057 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T10:42:22.058 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T10:42:22.059 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T10:42:22.059 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T10:42:22.057+0000 7f6cc1901700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 71606) UID: 1000 2021-03-09T10:42:22.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:42:25.060 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T10:42:25.060 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:42:25.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:42:25.061 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T10:42:25.061 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T10:42:25.062 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T10:42:25.064 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T10:42:25.065 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T10:42:51.047 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8786995315080807) > weight (0.25) 2021-03-09T10:42:51.047 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9420948480178429) > weight (0.25) 2021-03-09T10:42:51.048 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T10:42:51.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T10:42:51.049 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T10:42:51.049 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T10:42:51.044+0000 7f75e25da700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 71706) UID: 1000 2021-03-09T10:42:51.250 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:42:53.050 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T10:42:53.050 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:42:53.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:42:53.051 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T10:42:53.051 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T10:42:53.052 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T10:42:53.055 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T10:42:53.055 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T10:43:31.066 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32505078397881215) > weight (0.25) 2021-03-09T10:43:31.067 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9937648688574456) > weight (0.25) 2021-03-09T10:43:31.068 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4996727071313686) > weight (0.25) 2021-03-09T10:43:31.068 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9697495175282823) > weight (0.25) 2021-03-09T10:43:31.068 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T10:44:15.056 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.37551699421408136) > weight (0.25) 2021-03-09T10:44:15.059 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T10:44:15.060 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T10:44:15.061 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T10:44:15.062 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T10:44:15.055+0000 7efcd4ab5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 71515) UID: 1000 2021-03-09T10:44:15.262 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:44:21.062 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T10:44:21.062 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:44:21.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:44:21.063 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T10:44:21.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T10:44:21.063 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T10:44:21.066 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T10:44:21.066 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T10:45:28.070 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T10:45:28.071 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T10:45:28.072 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T10:45:28.072 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T10:45:28.066+0000 7f5de1816700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 71890) UID: 1000 2021-03-09T10:45:28.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:45:34.073 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T10:45:34.073 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:45:34.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:45:34.074 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T10:45:34.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T10:45:34.075 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T10:45:34.079 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T10:45:34.080 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T10:46:18.067 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T10:46:18.071 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T10:46:18.072 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T10:46:18.073 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T10:46:18.065+0000 7f28028e7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 71798) UID: 1000 2021-03-09T10:46:18.273 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:46:22.073 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T10:46:22.073 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:46:22.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:46:22.074 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T10:46:22.074 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T10:46:22.075 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T10:46:22.078 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T10:46:22.078 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T10:47:24.081 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6616462425011896) > weight (0.25) 2021-03-09T10:47:24.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5338150868235118) > weight (0.25) 2021-03-09T10:47:24.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6961127448007884) > weight (0.25) 2021-03-09T10:47:24.084 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T10:47:24.085 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T10:47:24.086 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T10:47:24.087 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T10:47:24.087 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:47:24.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T10:47:24.084+0000 7fb3fd7bb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 69217) UID: 1000 2021-03-09T10:47:24.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:47:28.079 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.37186862035558166) > weight (0.25) 2021-03-09T10:47:28.079 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7537933405801729) > weight (0.25) 2021-03-09T10:47:28.080 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T10:47:28.080 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T10:47:28.080 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T10:47:28.081 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T10:47:28.076+0000 7f94e6669700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 71990) UID: 1000 2021-03-09T10:47:28.282 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:47:30.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:47:30.190 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T10:47:30.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T10:47:30.191 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T10:47:30.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T10:47:30.194 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T10:47:32.203 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T10:47:32.197+0000 7fdda4a8d700 -1 reset not still connected to 0x55af003d18e0 2021-03-09T10:47:36.082 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T10:47:36.082 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:47:36.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:47:36.084 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T10:47:36.084 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T10:47:36.085 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T10:47:36.088 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T10:47:36.089 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-03-09T10:49:11.090 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T10:49:11.094 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T10:49:11.094 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T10:49:11.095 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T10:49:11.090+0000 7fd595170700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 72275) UID: 1000 2021-03-09T10:49:11.297 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:49:15.195 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9286400271617123) > weight (0.25) 2021-03-09T10:49:15.195 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T10:49:15.196 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T10:49:15.196 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T10:49:15.197 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T10:49:15.190+0000 7f100735b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 71413) UID: 1000 2021-03-09T10:49:15.398 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:49:16.197 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T10:49:16.197 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:49:16.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:49:16.198 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T10:49:16.198 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T10:49:16.199 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T10:49:16.202 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T10:49:16.202 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T10:49:18.096 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T10:49:18.096 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:49:18.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:49:18.097 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T10:49:18.097 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T10:49:18.098 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T10:49:18.101 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T10:49:18.101 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T10:50:38.204 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7627773273558578) > weight (0.25) 2021-03-09T10:50:38.205 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7361716994189682) > weight (0.25) 2021-03-09T10:50:38.205 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.708073053092838) > weight (0.25) 2021-03-09T10:50:38.205 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9611230959103119) > weight (0.25) 2021-03-09T10:50:38.205 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T10:50:49.102 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6666434383791341) > weight (0.25) 2021-03-09T10:50:49.107 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4978713386011707) > weight (0.25) 2021-03-09T10:50:49.107 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9335189053824766) > weight (0.25) 2021-03-09T10:50:49.107 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T10:50:49.109 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T10:50:49.109 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T10:50:49.111 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T10:50:49.108+0000 7fefee953700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 71119) UID: 1000 2021-03-09T10:50:49.511 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:50:50.292 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:50:50.288+0000 7f082a074700 -1 reset not still connected to 0x55a99ea04cb0 2021-03-09T10:50:51.111 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T10:50:51.112 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:50:51.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:50:51.112 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T10:50:51.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T10:50:51.113 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T10:50:51.116 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T10:50:51.116 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T10:51:38.206 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5613111874495545) > weight (0.25) 2021-03-09T10:51:38.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6844751034749676) > weight (0.25) 2021-03-09T10:51:38.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8253250636599359) > weight (0.25) 2021-03-09T10:51:38.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6143763362328877) > weight (0.25) 2021-03-09T10:51:38.212 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T10:52:35.117 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.39795869289048313) > weight (0.25) 2021-03-09T10:52:35.118 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2651722385633788) > weight (0.25) 2021-03-09T10:52:35.118 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5385207237423034) > weight (0.25) 2021-03-09T10:52:35.119 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T10:52:35.120 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T10:52:35.120 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T10:52:35.121 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T10:52:35.115+0000 7fa809bce700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 72756) UID: 1000 2021-03-09T10:52:35.321 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:52:41.121 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T10:52:41.121 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:52:41.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:52:41.122 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T10:52:41.123 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T10:52:41.123 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T10:52:41.126 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T10:52:41.127 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T10:53:32.213 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6152408228463128) > weight (0.25) 2021-03-09T10:53:32.214 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6931694845038869) > weight (0.25) 2021-03-09T10:53:32.214 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4222795853509508) > weight (0.25) 2021-03-09T10:53:32.214 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T10:53:32.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T10:53:32.215 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T10:53:32.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T10:53:32.214+0000 7f82bbf3a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 72367) UID: 1000 2021-03-09T10:53:32.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:53:40.216 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T10:53:40.216 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:53:40.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:53:40.217 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T10:53:40.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T10:53:40.218 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T10:53:40.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T10:53:40.222 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T10:54:09.128 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8170769768032481) > weight (0.25) 2021-03-09T10:54:09.130 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T10:54:09.131 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T10:54:09.131 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T10:54:09.132 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T10:54:09.129+0000 7f4467b03700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 72081) UID: 1000 2021-03-09T10:54:09.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:54:14.132 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T10:54:14.133 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:54:14.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:54:14.134 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T10:54:14.134 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T10:54:14.135 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T10:54:14.137 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T10:54:14.138 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T10:55:27.223 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T10:55:27.227 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T10:55:27.228 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T10:55:27.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T10:55:27.224+0000 7fe0e12f0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 72176) UID: 1000 2021-03-09T10:55:27.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:55:34.229 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T10:55:34.229 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:55:34.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:55:34.230 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T10:55:34.231 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T10:55:34.231 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T10:55:34.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T10:55:34.234 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T10:55:57.811 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T10:55:57.803+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad0227a80 2021-03-09T10:55:57.811 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T10:55:57.803+0000 7fa08ad3c700 -1 reset not still connected to 0x564adcac0150 2021-03-09T10:55:57.812 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T10:55:57.807+0000 7fa08ad3c700 -1 reset not still connected to 0x564adcac1260 2021-03-09T10:56:07.139 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.44605116942040046) > weight (0.25) 2021-03-09T10:56:07.139 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8800434674399125) > weight (0.25) 2021-03-09T10:56:07.140 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T10:56:07.140 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T10:56:07.141 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T10:56:07.142 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T10:56:07.135+0000 7fd7b6d89700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 72458) UID: 1000 2021-03-09T10:56:07.342 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:56:16.142 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T10:56:16.143 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:56:16.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:56:16.143 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T10:56:16.144 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T10:56:16.144 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T10:56:16.147 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T10:56:16.148 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T10:56:54.236 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6847034889055218) > weight (0.25) 2021-03-09T10:56:54.239 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5747356994979644) > weight (0.25) 2021-03-09T10:56:54.239 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T10:56:54.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T10:56:54.242 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T10:56:54.243 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T10:56:54.238+0000 7f1cbcb70700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 69883) UID: 1000 2021-03-09T10:56:54.642 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:56:56.243 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T10:56:56.244 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:56:56.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:56:56.245 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T10:56:56.245 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T10:56:56.245 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T10:56:56.248 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T10:56:56.249 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T10:57:48.149 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5631998332337298) > weight (0.25) 2021-03-09T10:57:48.149 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T10:57:48.150 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T10:57:48.150 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T10:57:48.151 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T10:57:48.145+0000 7f41b8910700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 73039) UID: 1000 2021-03-09T10:57:48.351 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:57:55.152 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T10:57:55.152 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:57:55.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:57:55.153 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T10:57:55.153 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T10:57:55.153 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T10:57:55.157 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T10:57:55.157 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T10:57:59.250 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6351850255571134) > weight (0.25) 2021-03-09T10:57:59.250 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3473578539471348) > weight (0.25) 2021-03-09T10:57:59.251 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T10:57:59.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T10:57:59.251 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T10:57:59.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T10:57:59.249+0000 7fce828da700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 73324) UID: 1000 2021-03-09T10:57:59.453 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:58:07.253 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T10:58:07.253 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:58:07.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:58:07.254 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T10:58:07.254 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T10:58:07.255 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T10:58:07.258 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T10:58:07.258 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T10:58:46.407 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:58:46.400+0000 7f082a074700 -1 reset not still connected to 0x55a996498080 2021-03-09T10:58:46.407 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:58:46.400+0000 7f082a074700 -1 reset not still connected to 0x55a99af2e630 2021-03-09T10:58:46.408 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T10:58:46.400+0000 7f082a074700 -1 reset not still connected to 0x55aa09b2a340 2021-03-09T10:59:09.259 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.890365931899929) > weight (0.25) 2021-03-09T10:59:09.260 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7891211177561858) > weight (0.25) 2021-03-09T10:59:09.260 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6654856513862274) > weight (0.25) 2021-03-09T10:59:09.260 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T10:59:09.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T10:59:09.262 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T10:59:09.265 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T10:59:09.260+0000 7f9ef9427700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 72940) UID: 1000 2021-03-09T10:59:09.465 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:59:18.263 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T10:59:18.263 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:59:18.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:59:18.264 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T10:59:18.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T10:59:18.265 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T10:59:18.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T10:59:18.268 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T10:59:32.158 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4254559839261639) > weight (0.25) 2021-03-09T10:59:32.159 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8188630661669378) > weight (0.25) 2021-03-09T10:59:32.159 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.861070617447139) > weight (0.25) 2021-03-09T10:59:32.159 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T10:59:32.160 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T10:59:32.160 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T10:59:32.161 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T10:59:32.156+0000 7fb1bb6c5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 72847) UID: 1000 2021-03-09T10:59:32.362 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T10:59:36.161 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T10:59:36.162 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T10:59:36.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T10:59:36.163 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T10:59:36.163 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T10:59:36.163 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T10:59:36.166 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T10:59:36.166 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T11:00:31.269 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T11:00:31.271 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T11:00:31.271 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T11:00:31.272 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:00:31.267+0000 7fa553d65700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 73133) UID: 1000 2021-03-09T11:00:31.343 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T11:00:31.339+0000 7fa08ad3c700 -1 reset not still connected to 0x564add859400 2021-03-09T11:00:31.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:00:38.272 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T11:00:38.272 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:00:38.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:00:38.273 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T11:00:38.274 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T11:00:38.274 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T11:00:38.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T11:00:38.278 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T11:01:20.167 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3466144827000389) > weight (0.25) 2021-03-09T11:01:20.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.515731199463191) > weight (0.25) 2021-03-09T11:01:20.168 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3462994656354952) > weight (0.25) 2021-03-09T11:01:20.168 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T11:01:20.169 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T11:01:20.170 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T11:01:20.170 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T11:01:20.170 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:01:20.171 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T11:01:20.166+0000 7f71ef820700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 73715) UID: 1000 2021-03-09T11:01:20.371 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:01:26.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:01:26.272 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T11:01:26.273 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T11:01:26.273 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T11:01:26.277 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T11:01:26.277 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T11:02:06.279 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T11:02:06.281 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T11:02:06.281 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T11:02:06.282 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:02:06.273+0000 7f65abb07700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 73808) UID: 1000 2021-03-09T11:02:06.482 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:02:12.282 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T11:02:12.283 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:02:12.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:02:12.283 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T11:02:12.284 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T11:02:12.284 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T11:02:12.287 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T11:02:12.287 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T11:02:57.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.34796733563347293) > weight (0.25) 2021-03-09T11:02:57.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.72660690658814) > weight (0.25) 2021-03-09T11:02:57.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.28857350901111245) > weight (0.25) 2021-03-09T11:02:57.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4763830103022294) > weight (0.25) 2021-03-09T11:02:57.279 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-03-09T11:03:59.288 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3017830496149053) > weight (0.25) 2021-03-09T11:03:59.289 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T11:03:59.289 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T11:03:59.290 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T11:03:59.291 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:03:59.287+0000 7f582f30c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 72559) UID: 1000 2021-03-09T11:03:59.491 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:04:00.502 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:04:00.493+0000 7f901a51c700 -1 reset not still connected to 0x5588fb945ee0 2021-03-09T11:04:00.502 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:04:00.493+0000 7f901a51c700 -1 reset not still connected to 0x558904591c70 2021-03-09T11:04:00.505 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:04:00.497+0000 7f901a51c700 -1 reset not still connected to 0x558904592970 2021-03-09T11:04:00.547 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:04:00.541+0000 7f901a51c700 -1 reset not still connected to 0x5588fb945ee0 2021-03-09T11:04:00.547 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:04:00.541+0000 7f901a51c700 -1 reset not still connected to 0x558904592970 2021-03-09T11:04:02.291 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T11:04:02.291 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:04:02.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:04:02.292 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T11:04:02.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T11:04:02.293 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T11:04:02.296 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T11:04:02.296 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T11:04:05.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.38475494852528025) > weight (0.25) 2021-03-09T11:04:05.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.29862130445949153) > weight (0.25) 2021-03-09T11:04:05.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3527260181058274) > weight (0.25) 2021-03-09T11:04:05.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8765923038257273) > weight (0.25) 2021-03-09T11:04:05.282 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T11:05:29.283 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6169180000214869) > weight (0.25) 2021-03-09T11:05:29.285 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.39569391071029736) > weight (0.25) 2021-03-09T11:05:29.285 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T11:05:29.286 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T11:05:29.287 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T11:05:29.288 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T11:05:29.281+0000 7f79eb2f7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 73232) UID: 1000 2021-03-09T11:05:29.488 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:05:31.288 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T11:05:31.288 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:05:31.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:05:31.289 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T11:05:31.289 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T11:05:31.290 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T11:05:31.293 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T11:05:31.293 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T11:05:41.297 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.387896732559211) > weight (0.25) 2021-03-09T11:05:41.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.884175425420787) > weight (0.25) 2021-03-09T11:05:41.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6246297102108431) > weight (0.25) 2021-03-09T11:05:41.299 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8373688016332448) > weight (0.25) 2021-03-09T11:05:41.300 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T11:07:01.294 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7646426847167224) > weight (0.25) 2021-03-09T11:07:01.296 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5663322050462305) > weight (0.25) 2021-03-09T11:07:01.296 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T11:07:01.297 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T11:07:01.297 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T11:07:01.298 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T11:07:01.292+0000 7f6df51fe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 74203) UID: 1000 2021-03-09T11:07:01.500 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:07:09.298 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T11:07:09.301 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:07:09.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:07:09.302 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T11:07:09.302 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T11:07:09.303 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T11:07:09.306 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T11:07:09.306 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-03-09T11:07:30.301 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T11:07:30.302 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T11:07:30.302 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T11:07:30.302 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T11:07:30.303 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:07:30.303 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:07:30.299+0000 7f7bcd75c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 73999) UID: 1000 2021-03-09T11:07:30.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:07:36.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:07:36.406 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T11:07:36.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T11:07:36.407 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T11:07:36.409 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T11:07:36.410 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T11:08:17.307 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.49977627760803056) > weight (0.25) 2021-03-09T11:08:17.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.41222793409279346) > weight (0.25) 2021-03-09T11:08:17.308 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.32622869249249875) > weight (0.25) 2021-03-09T11:08:17.309 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9724336278971156) > weight (0.25) 2021-03-09T11:08:17.309 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T11:09:18.310 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8333191097662094) > weight (0.25) 2021-03-09T11:09:18.317 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.44312636836635244) > weight (0.25) 2021-03-09T11:09:18.317 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9884971577351321) > weight (0.25) 2021-03-09T11:09:18.317 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.31871554846450145) > weight (0.25) 2021-03-09T11:09:18.317 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T11:09:26.414 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T11:09:26.418 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T11:09:26.419 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T11:09:26.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:09:26.413+0000 7fd8a4dee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 74387) UID: 1000 2021-03-09T11:09:26.619 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:09:32.423 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T11:09:32.424 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:09:32.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:09:32.425 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T11:09:32.425 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T11:09:32.431 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T11:09:32.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T11:09:32.435 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T11:10:40.318 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6409847924603032) > weight (0.25) 2021-03-09T11:10:40.325 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3297217345954766) > weight (0.25) 2021-03-09T11:10:40.325 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T11:10:40.326 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T11:10:40.329 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T11:10:40.330 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T11:10:40.324+0000 7f8787359700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 74296) UID: 1000 2021-03-09T11:10:40.528 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:10:42.330 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T11:10:42.331 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:10:42.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:10:42.332 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T11:10:42.332 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T11:10:42.437 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T11:10:42.442 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T11:10:42.442 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T11:11:01.436 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3044983653305515) > weight (0.25) 2021-03-09T11:11:01.441 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6414601389748145) > weight (0.25) 2021-03-09T11:11:01.441 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4398545393869908) > weight (0.25) 2021-03-09T11:11:01.442 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9308891158834125) > weight (0.25) 2021-03-09T11:11:01.442 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T11:12:14.443 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48510994819341113) > weight (0.25) 2021-03-09T11:12:14.445 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5251556891083627) > weight (0.25) 2021-03-09T11:12:14.445 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T11:12:14.446 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T11:12:14.447 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T11:12:14.448 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T11:12:14.443+0000 7f5acf58a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 73514) UID: 1000 2021-03-09T11:12:14.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:12:17.448 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T11:12:17.449 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:12:17.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:12:17.450 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T11:12:17.450 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T11:12:17.450 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T11:12:17.454 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T11:12:17.454 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T11:12:26.443 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9110441407192604) > weight (0.25) 2021-03-09T11:12:26.444 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7497620031129323) > weight (0.25) 2021-03-09T11:12:26.445 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8634251660702946) > weight (0.25) 2021-03-09T11:12:26.445 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T11:12:26.446 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T11:12:26.446 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T11:12:26.447 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T11:12:26.442+0000 7f6c7323a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 73907) UID: 1000 2021-03-09T11:12:26.647 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:12:31.447 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T11:12:31.448 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:12:31.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:12:31.449 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T11:12:31.449 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T11:12:31.449 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T11:12:31.452 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T11:12:31.452 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-03-09T11:12:41.964 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:12:41.954+0000 7f901a51c700 -1 reset not still connected to 0x5588f923f9b0 2021-03-09T11:12:41.965 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:12:41.954+0000 7f901a51c700 -1 reset not still connected to 0x5588fb946150 2021-03-09T11:12:41.965 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:12:41.954+0000 7f901a51c700 -1 reset not still connected to 0x5588fb947190 2021-03-09T11:12:41.965 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:12:41.958+0000 7f901a51c700 -1 reset not still connected to 0x558904591d40 2021-03-09T11:12:45.006 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:12:45.002+0000 7f901a51c700 -1 reset not still connected to 0x5588fb59bb50 2021-03-09T11:12:45.008 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:12:45.002+0000 7f901a51c700 -1 reset not still connected to 0x5588ffe7bba0 2021-03-09T11:13:37.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7354496677107173) > weight (0.25) 2021-03-09T11:13:37.456 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.39609371991338416) > weight (0.25) 2021-03-09T11:13:37.456 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.44955541751297456) > weight (0.25) 2021-03-09T11:13:37.456 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5678592229060082) > weight (0.25) 2021-03-09T11:13:37.457 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T11:13:57.454 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.31626314671966194) > weight (0.25) 2021-03-09T11:13:57.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5884079956772047) > weight (0.25) 2021-03-09T11:13:57.455 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5310942032384466) > weight (0.25) 2021-03-09T11:13:57.455 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T11:13:57.456 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T11:13:57.456 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T11:13:57.458 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T11:13:57.453+0000 7f89e27b6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 74778) UID: 1000 2021-03-09T11:13:57.658 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:13:58.458 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T11:13:58.458 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:13:58.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:13:58.459 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T11:13:58.459 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T11:13:58.460 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T11:13:58.463 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T11:13:58.463 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-03-09T11:15:19.464 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3705243196820621) > weight (0.25) 2021-03-09T11:15:19.499 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T11:15:19.500 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T11:15:19.500 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T11:15:19.501 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:15:19.495+0000 7f0e45899700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 73423) UID: 1000 2021-03-09T11:15:19.702 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:15:25.501 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T11:15:25.501 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:15:25.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:15:25.502 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T11:15:25.503 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T11:15:25.503 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T11:15:25.527 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T11:15:25.527 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T11:15:33.457 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7049110749338167) > weight (0.25) 2021-03-09T11:15:33.458 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.40089073020642363) > weight (0.25) 2021-03-09T11:15:33.458 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.259116851265651) > weight (0.25) 2021-03-09T11:15:33.459 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4177374300416492) > weight (0.25) 2021-03-09T11:15:33.459 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T11:16:12.657 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T11:16:12.651+0000 7f082a074700 -1 reset not still connected to 0x55a9a13aa0d0 2021-03-09T11:16:31.528 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T11:16:31.529 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T11:16:31.529 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T11:16:31.530 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T11:16:31.530 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:16:31.531 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T11:16:31.526+0000 7faeb3eea700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 72650) UID: 1000 2021-03-09T11:16:31.732 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:16:35.461 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6804075829385378) > weight (0.25) 2021-03-09T11:16:35.461 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5822696015421536) > weight (0.25) 2021-03-09T11:16:35.461 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.552324396086383) > weight (0.25) 2021-03-09T11:16:35.462 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4913037079433977) > weight (0.25) 2021-03-09T11:16:35.462 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T11:16:37.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:16:37.632 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T11:16:37.633 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T11:16:37.633 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T11:16:37.636 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T11:16:37.636 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T11:17:39.637 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7971425188805209) > weight (0.25) 2021-03-09T11:17:39.638 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T11:17:39.639 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T11:17:39.639 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T11:17:39.640 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:17:39.633+0000 7f9934761700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 74966) UID: 1000 2021-03-09T11:17:39.841 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:17:44.640 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T11:17:44.641 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:17:44.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:17:44.642 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T11:17:44.642 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T11:17:44.642 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T11:17:44.645 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T11:17:44.646 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T11:18:09.815 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:18:09.809+0000 7f98a4754700 -1 rbd::mirror::ImageReplayer: 0x55724634a080 [3/35b68617-078e-413b-9699-cf8f31934816] start: already running 2021-03-09T11:18:09.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:18:09.809+0000 7f98a4754700 -1 rbd::mirror::ImageReplayer: 0x55724634a300 [3/6cd921d3-989d-483b-ba40-00f209ebbd60] start: already running 2021-03-09T11:18:09.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:18:09.809+0000 7f98a4754700 -1 rbd::mirror::ImageReplayer: 0x55724634aa80 [3/6eb851a0-fa35-4de2-b7b1-277c2544ef7a] start: already running 2021-03-09T11:18:09.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:18:09.809+0000 7f98a4f55700 -1 rbd::mirror::ImageReplayer: 0x55724634a580 [3/b2b08618-0c38-4fe7-92b1-937ad9c546bf] start: already running 2021-03-09T11:18:32.463 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6002108913655569) > weight (0.25) 2021-03-09T11:18:32.489 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7134440319965399) > weight (0.25) 2021-03-09T11:18:32.489 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5152095584151324) > weight (0.25) 2021-03-09T11:18:32.489 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4766893769474492) > weight (0.25) 2021-03-09T11:18:32.490 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T11:18:50.646 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.706883176693738) > weight (0.25) 2021-03-09T11:18:50.647 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.57025657739697) > weight (0.25) 2021-03-09T11:18:50.647 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T11:18:50.648 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T11:18:50.648 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T11:18:50.649 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T11:18:50.644+0000 7fb79e8e7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 74588) UID: 1000 2021-03-09T11:18:50.850 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:18:59.650 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T11:18:59.650 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:18:59.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:18:59.651 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T11:18:59.651 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T11:18:59.652 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T11:18:59.655 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T11:18:59.655 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T11:20:15.491 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.963541598914669) > weight (0.25) 2021-03-09T11:20:15.494 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.31541873669404785) > weight (0.25) 2021-03-09T11:20:15.494 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3385004595855988) > weight (0.25) 2021-03-09T11:20:15.494 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7088222304177325) > weight (0.25) 2021-03-09T11:20:15.495 INFO:tasks.rbd_mirror_thrash:waiting for 100 secs before thrashing 2021-03-09T11:20:51.656 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8384632862899504) > weight (0.25) 2021-03-09T11:20:51.657 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9682205688102511) > weight (0.25) 2021-03-09T11:20:51.657 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6179690301584706) > weight (0.25) 2021-03-09T11:20:51.657 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7276060008871192) > weight (0.25) 2021-03-09T11:20:51.658 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T11:21:55.496 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48680949742766955) > weight (0.25) 2021-03-09T11:21:55.497 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8099254439186209) > weight (0.25) 2021-03-09T11:21:55.498 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.37354736330158456) > weight (0.25) 2021-03-09T11:21:55.498 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T11:21:55.499 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T11:21:55.499 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T11:21:55.500 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T11:21:55.497+0000 7f9d2fc67700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 73616) UID: 1000 2021-03-09T11:21:55.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:21:56.891 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T11:21:56.877+0000 7f082a074700 -1 reset not still connected to 0x55a9b3e3a560 2021-03-09T11:22:02.500 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T11:22:02.501 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:22:02.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:22:02.502 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T11:22:02.502 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T11:22:02.502 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T11:22:02.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T11:22:02.506 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T11:22:16.659 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.622063545969567) > weight (0.25) 2021-03-09T11:22:16.661 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7354122107216631) > weight (0.25) 2021-03-09T11:22:16.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.33356981509384886) > weight (0.25) 2021-03-09T11:22:16.662 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3689697707327101) > weight (0.25) 2021-03-09T11:22:16.662 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T11:23:23.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.430525910109281) > weight (0.25) 2021-03-09T11:23:23.665 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25084271489658283) > weight (0.25) 2021-03-09T11:23:23.665 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9919795842958161) > weight (0.25) 2021-03-09T11:23:23.665 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6734674828185176) > weight (0.25) 2021-03-09T11:23:23.666 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T11:23:27.507 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4801105410715504) > weight (0.25) 2021-03-09T11:23:27.507 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2818675104607614) > weight (0.25) 2021-03-09T11:23:27.507 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T11:23:27.508 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T11:23:27.509 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T11:23:27.509 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T11:23:27.507+0000 7f1ee90d1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 74679) UID: 1000 2021-03-09T11:23:27.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:23:33.510 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T11:23:33.511 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:23:33.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:23:33.512 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T11:23:33.512 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T11:23:33.514 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T11:23:33.517 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T11:23:33.517 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T11:25:02.519 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9681270653450403) > weight (0.25) 2021-03-09T11:25:02.533 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2621832916942275) > weight (0.25) 2021-03-09T11:25:02.534 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9178940755092361) > weight (0.25) 2021-03-09T11:25:02.534 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T11:25:02.535 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T11:25:02.536 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T11:25:02.537 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T11:25:02.534+0000 7f150af36700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 75338) UID: 1000 2021-03-09T11:25:02.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:25:08.666 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6826880272321317) > weight (0.25) 2021-03-09T11:25:08.667 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9197402737412912) > weight (0.25) 2021-03-09T11:25:08.667 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T11:25:08.668 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T11:25:08.668 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T11:25:08.669 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T11:25:08.661+0000 7f20d47e8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 75243) UID: 1000 2021-03-09T11:25:08.869 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:25:09.537 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T11:25:09.538 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:25:09.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:25:09.539 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T11:25:09.539 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T11:25:09.540 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T11:25:09.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T11:25:09.543 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T11:25:16.669 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T11:25:16.672 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:25:16.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:25:16.673 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T11:25:16.673 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T11:25:16.674 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T11:25:16.677 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T11:25:16.678 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T11:26:16.544 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6888889164419414) > weight (0.25) 2021-03-09T11:26:16.544 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9709463870035129) > weight (0.25) 2021-03-09T11:26:16.544 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7221633819508909) > weight (0.25) 2021-03-09T11:26:16.545 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T11:26:16.546 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T11:26:16.546 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T11:26:16.547 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T11:26:16.544+0000 7f4b5c303700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 75540) UID: 1000 2021-03-09T11:26:16.748 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:26:24.547 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T11:26:24.548 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:26:24.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:26:24.548 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T11:26:24.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T11:26:24.549 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T11:26:24.552 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T11:26:24.552 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T11:26:50.679 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T11:26:50.681 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T11:26:50.681 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T11:26:50.683 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T11:26:50.680+0000 7fdfa1c0c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 75057) UID: 1000 2021-03-09T11:26:50.882 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:26:54.682 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T11:26:54.683 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:26:54.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:26:54.684 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T11:26:54.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T11:26:54.684 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T11:26:54.687 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T11:26:54.688 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T11:27:24.553 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9227271767513102) > weight (0.25) 2021-03-09T11:27:24.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.721866721056287) > weight (0.25) 2021-03-09T11:27:24.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.452460339408168) > weight (0.25) 2021-03-09T11:27:24.555 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.28847344199016) > weight (0.25) 2021-03-09T11:27:24.555 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T11:27:55.689 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6987633066888451) > weight (0.25) 2021-03-09T11:27:55.691 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7839347936010878) > weight (0.25) 2021-03-09T11:27:55.691 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.537147138196091) > weight (0.25) 2021-03-09T11:27:55.691 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9908905385084357) > weight (0.25) 2021-03-09T11:27:55.691 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T11:28:33.557 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7902559237299599) > weight (0.25) 2021-03-09T11:28:33.559 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T11:28:33.560 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T11:28:33.561 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T11:28:33.562 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T11:28:33.562 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:28:33.563 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:28:33.558+0000 7f6765a40700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 74099) UID: 1000 2021-03-09T11:28:33.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:28:39.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:28:39.664 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T11:28:39.665 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T11:28:39.665 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T11:28:39.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T11:28:39.668 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T11:29:24.692 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6035232007057619) > weight (0.25) 2021-03-09T11:29:24.696 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T11:29:24.697 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T11:29:24.697 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T11:29:24.698 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:29:24.689+0000 7f98a5756700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 75151) UID: 1000 2021-03-09T11:29:24.898 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:29:27.698 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T11:29:27.699 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:29:27.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:29:27.700 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T11:29:27.700 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T11:29:27.700 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T11:29:27.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T11:29:27.703 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T11:30:13.669 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3370784166429822) > weight (0.25) 2021-03-09T11:30:13.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6716841492580339) > weight (0.25) 2021-03-09T11:30:13.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9763820457433662) > weight (0.25) 2021-03-09T11:30:13.670 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5199205111544033) > weight (0.25) 2021-03-09T11:30:13.671 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T11:31:26.705 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8171217798559562) > weight (0.25) 2021-03-09T11:31:26.747 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T11:31:26.748 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T11:31:26.748 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T11:31:26.749 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:31:26.743+0000 7f35dcb01700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 76021) UID: 1000 2021-03-09T11:31:26.950 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:31:34.749 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T11:31:34.749 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:31:34.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:31:34.750 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T11:31:34.751 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T11:31:34.751 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T11:31:34.755 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T11:31:34.756 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T11:31:34.953 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:31:34.945+0000 7f901a51c700 -1 reset not still connected to 0x558900715670 2021-03-09T11:31:34.953 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:31:34.945+0000 7f901a51c700 -1 reset not still connected to 0x55890173bad0 2021-03-09T11:31:34.953 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:31:34.945+0000 7f901a51c700 -1 reset not still connected to 0x558901884220 2021-03-09T11:31:34.954 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:31:34.945+0000 7f901a51c700 -1 reset not still connected to 0x558901884970 2021-03-09T11:32:02.672 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9230238885077517) > weight (0.25) 2021-03-09T11:32:02.672 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.702672255465227) > weight (0.25) 2021-03-09T11:32:02.673 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.45774992967937267) > weight (0.25) 2021-03-09T11:32:02.673 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T11:32:02.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T11:32:02.674 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T11:32:02.675 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T11:32:02.670+0000 7fcb97ceb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 75732) UID: 1000 2021-03-09T11:32:02.875 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:32:10.675 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T11:32:10.675 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:32:10.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:32:10.676 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T11:32:10.677 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T11:32:10.677 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T11:32:10.680 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T11:32:10.680 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T11:32:35.001 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T11:32:34.994+0000 7f082a074700 -1 reset not still connected to 0x55a99a1af520 2021-03-09T11:32:35.001 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T11:32:34.994+0000 7f082a074700 -1 reset not still connected to 0x55a99af2e700 2021-03-09T11:32:35.001 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T11:32:34.994+0000 7f082a074700 -1 reset not still connected to 0x55a9a2305c20 2021-03-09T11:32:40.756 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9430786190822729) > weight (0.25) 2021-03-09T11:32:40.762 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T11:32:40.764 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T11:32:40.764 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T11:32:40.768 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:32:40.762+0000 7f5efa130700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 76114) UID: 1000 2021-03-09T11:32:40.968 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:32:41.765 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T11:32:41.766 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:32:41.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:32:41.767 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T11:32:41.767 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T11:32:41.767 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T11:32:41.773 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T11:32:41.773 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T11:33:39.681 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7309110369580755) > weight (0.25) 2021-03-09T11:33:39.682 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T11:33:39.683 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T11:33:39.683 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T11:33:39.684 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:33:39.681+0000 7fe8cd575700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 75922) UID: 1000 2021-03-09T11:33:39.885 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:33:40.684 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T11:33:40.685 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:33:40.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:33:40.686 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T11:33:40.686 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T11:33:40.686 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T11:33:40.689 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T11:33:40.690 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T11:33:46.775 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T11:33:46.776 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T11:33:46.776 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T11:33:46.777 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T11:33:46.773+0000 7f91ae749700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 75831) UID: 1000 2021-03-09T11:33:46.977 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:33:48.777 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T11:33:48.778 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:33:48.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:33:48.779 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T11:33:48.779 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T11:33:48.780 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T11:33:48.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T11:33:48.782 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-03-09T11:35:03.783 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.605984655893607) > weight (0.25) 2021-03-09T11:35:03.787 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6555974500168029) > weight (0.25) 2021-03-09T11:35:03.787 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T11:35:03.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T11:35:03.789 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T11:35:03.790 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T11:35:03.787+0000 7f344b4ed700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 75639) UID: 1000 2021-03-09T11:35:03.990 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:35:04.691 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7693159418263713) > weight (0.25) 2021-03-09T11:35:04.691 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6522764374824798) > weight (0.25) 2021-03-09T11:35:04.691 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7142359884850223) > weight (0.25) 2021-03-09T11:35:04.692 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T11:35:04.692 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T11:35:04.692 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T11:35:04.693 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T11:35:04.691+0000 7f0a32c8e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 76207) UID: 1000 2021-03-09T11:35:04.894 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:35:09.694 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T11:35:09.694 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:35:09.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:35:09.695 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T11:35:09.695 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T11:35:09.696 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T11:35:09.698 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T11:35:09.699 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T11:35:12.790 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T11:35:12.790 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:35:12.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:35:12.791 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T11:35:12.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T11:35:12.792 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T11:35:12.796 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T11:35:12.796 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T11:35:14.200 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:35:14.195+0000 7f901a51c700 -1 reset not still connected to 0x55890173ddc0 2021-03-09T11:35:14.200 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:35:14.195+0000 7f901a51c700 -1 reset not still connected to 0x558901882f70 2021-03-09T11:35:14.205 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:35:14.195+0000 7f901a51c700 -1 reset not still connected to 0x558904ab2150 2021-03-09T11:35:14.206 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:35:14.195+0000 7f901a51c700 -1 reset not still connected to 0x558904ab2490 2021-03-09T11:35:14.206 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T11:35:14.195+0000 7f901a51c700 -1 reset not still connected to 0x558904ab3e90 2021-03-09T11:36:34.700 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9188844560691019) > weight (0.25) 2021-03-09T11:36:34.701 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.27217108063004647) > weight (0.25) 2021-03-09T11:36:34.701 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T11:36:34.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T11:36:34.702 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T11:36:34.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T11:36:34.698+0000 7f51bdf1e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 75438) UID: 1000 2021-03-09T11:36:34.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:36:35.703 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T11:36:35.704 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:36:35.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:36:35.705 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T11:36:35.705 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T11:36:35.706 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T11:36:35.709 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T11:36:35.709 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T11:36:42.798 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.33812067971898774) > weight (0.25) 2021-03-09T11:36:42.798 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.31238844864851356) > weight (0.25) 2021-03-09T11:36:42.798 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5999494096065541) > weight (0.25) 2021-03-09T11:36:42.799 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3548476656984425) > weight (0.25) 2021-03-09T11:36:42.799 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T11:37:55.800 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.41589550826786137) > weight (0.25) 2021-03-09T11:37:55.804 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T11:37:55.805 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T11:37:55.805 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T11:37:55.806 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:37:55.804+0000 7f2d4bdc4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 76306) UID: 1000 2021-03-09T11:37:56.006 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:37:56.806 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T11:37:56.806 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:37:56.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:37:56.807 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T11:37:56.807 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T11:37:56.808 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T11:37:56.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T11:37:56.815 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T11:38:17.710 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8121448758926142) > weight (0.25) 2021-03-09T11:38:17.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7041527260433359) > weight (0.25) 2021-03-09T11:38:17.712 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6244665847587022) > weight (0.25) 2021-03-09T11:38:17.712 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T11:38:17.713 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T11:38:17.713 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T11:38:17.714 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T11:38:17.712+0000 7f4607d4f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 76591) UID: 1000 2021-03-09T11:38:17.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:38:25.715 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T11:38:25.715 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:38:25.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:38:25.716 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T11:38:25.716 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T11:38:25.717 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T11:38:25.720 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T11:38:25.720 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T11:39:28.816 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T11:39:28.818 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T11:39:28.819 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T11:39:28.820 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T11:39:28.815+0000 7eff2a53f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 76496) UID: 1000 2021-03-09T11:39:29.020 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:39:29.721 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T11:39:29.722 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T11:39:29.722 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T11:39:29.723 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:39:29.719+0000 7f2212125700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 74488) UID: 1000 2021-03-09T11:39:30.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:39:31.819 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T11:39:31.820 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:39:31.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:39:31.820 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T11:39:31.821 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T11:39:31.821 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T11:39:31.824 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T11:39:31.824 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-03-09T11:39:32.723 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T11:39:32.724 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:39:32.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:39:32.725 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T11:39:32.725 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T11:39:32.725 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T11:39:32.728 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T11:39:32.728 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T11:41:01.730 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2987253316639432) > weight (0.25) 2021-03-09T11:41:01.739 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T11:41:01.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T11:41:01.741 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T11:41:01.741 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:41:01.737+0000 7f8d7cad3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 76397) UID: 1000 2021-03-09T11:41:01.942 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:41:05.742 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T11:41:05.742 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:41:05.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:41:05.743 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T11:41:05.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T11:41:05.744 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T11:41:05.747 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T11:41:05.747 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T11:41:07.825 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.673499749237829) > weight (0.25) 2021-03-09T11:41:07.826 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T11:41:07.826 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T11:41:07.826 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T11:41:07.827 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:41:07.825+0000 7fb5bd7cd700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 76883) UID: 1000 2021-03-09T11:41:08.028 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:41:13.828 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T11:41:13.828 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:41:13.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:41:13.829 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T11:41:13.829 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T11:41:13.830 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T11:41:13.832 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T11:41:13.833 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T11:42:44.748 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3244426118748974) > weight (0.25) 2021-03-09T11:42:44.752 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5393282162551982) > weight (0.25) 2021-03-09T11:42:44.752 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4594578527341905) > weight (0.25) 2021-03-09T11:42:44.752 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5786675829235441) > weight (0.25) 2021-03-09T11:42:44.753 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-03-09T11:43:08.834 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7655472068667196) > weight (0.25) 2021-03-09T11:43:08.836 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T11:43:08.837 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T11:43:08.837 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T11:43:08.838 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:43:08.831+0000 7f24b93e3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 77355) UID: 1000 2021-03-09T11:43:09.038 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:43:11.838 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T11:43:11.838 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:43:11.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:43:11.839 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T11:43:11.839 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T11:43:11.840 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T11:43:11.843 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T11:43:11.844 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T11:44:16.845 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T11:44:16.848 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T11:44:16.849 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T11:44:16.849 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T11:44:16.842+0000 7f213396f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 77073) UID: 1000 2021-03-09T11:44:17.050 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:44:19.753 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5548809758864807) > weight (0.25) 2021-03-09T11:44:19.754 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8756533699500587) > weight (0.25) 2021-03-09T11:44:19.754 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T11:44:19.754 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T11:44:19.755 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T11:44:19.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T11:44:19.750+0000 7f3c2c5e6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 76781) UID: 1000 2021-03-09T11:44:20.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:44:20.756 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T11:44:20.756 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:44:20.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:44:20.757 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T11:44:20.757 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T11:44:20.758 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T11:44:20.760 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T11:44:20.761 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T11:44:20.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.926+0000 7f04423ce700 -1 rbd::mirror::ImageReplayer: 0x55e4d1087180 [3/9d177d94-7bc5-46be-8fff-f48edf3c1f9d] start: already running 2021-03-09T11:44:20.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.926+0000 7f04423ce700 -1 rbd::mirror::ImageReplayer: 0x55e4d1087b80 [3/aec0a6af-f0ca-401c-a5d1-8f250674262a] start: already running 2021-03-09T11:44:20.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.926+0000 7f04423ce700 -1 rbd::mirror::ImageReplayer: 0x55e4d1087900 [3/b1eacee4-665d-41b5-9f60-43674f116887] start: already running 2021-03-09T11:44:20.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.926+0000 7f04423ce700 -1 rbd::mirror::ImageReplayer: 0x55e4d1087e00 [3/b2b08618-0c38-4fe7-92b1-937ad9c546bf] start: already running 2021-03-09T11:44:20.932 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:44:20.926+0000 7f1af5cf0700 -1 rbd::mirror::ImageReplayer: 0x564091547900 [3/1a6951f1-4951-4047-8edb-1035961df315] start: already running 2021-03-09T11:44:20.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:44:20.926+0000 7f1af5cf0700 -1 rbd::mirror::ImageReplayer: 0x564091547b80 [3/35b68617-078e-413b-9699-cf8f31934816] start: already running 2021-03-09T11:44:20.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:44:20.926+0000 7f1af5cf0700 -1 rbd::mirror::ImageReplayer: 0x564091547180 [3/3b232a3b-57f6-42ac-8d1f-68890a58c5ed] start: already running 2021-03-09T11:44:20.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:44:20.926+0000 7f1af5cf0700 -1 rbd::mirror::ImageReplayer: 0x564091546f00 [3/5f5c4aa6-1fcb-40a3-83e8-e1123b50fd68] start: already running 2021-03-09T11:44:20.933 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:44:20.926+0000 7f1af5cf0700 -1 rbd::mirror::ImageReplayer: 0x564091548a80 [3/76a412aa-913f-4f54-ad5f-0dbad3c607cd] start: already running 2021-03-09T11:44:20.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:44:20.926+0000 7f1af5cf0700 -1 rbd::mirror::ImageReplayer: 0x564091547400 [3/8853c04a-4df4-47fb-9df9-6cebf19cc680] start: already running 2021-03-09T11:44:20.934 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:44:20.926+0000 7f1af5cf0700 -1 rbd::mirror::ImageReplayer: 0x564091548d00 [3/941e8ddf-64f0-4f25-89c8-9cfbbec2992f] start: already running 2021-03-09T11:44:20.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:44:20.930+0000 7f1af64f1700 -1 rbd::mirror::ImageReplayer: 0x564091549200 [3/cbb866ff-7e4b-4de9-80d6-4f875228d852] start: already running 2021-03-09T11:44:20.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:44:20.930+0000 7f1af64f1700 -1 rbd::mirror::ImageReplayer: 0x564091548f80 [3/f250f852-7768-44dd-84f7-ce3818998e65] start: already running 2021-03-09T11:44:20.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.930+0000 7f04423ce700 -1 rbd::mirror::ImageReplayer: 0x55e4d1087680 [3/82fd07ce-ce87-4a9a-8826-1102d2c5803e] start: already running 2021-03-09T11:44:20.939 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.934+0000 7f04423ce700 -1 rbd::mirror::ImageReplayer: 0x55e4d1086f00 [3/8f8bd0be-65f9-414f-94b0-b7682eaded5e] start: already running 2021-03-09T11:44:20.941 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.934+0000 7f04423ce700 -1 rbd::mirror::ImageReplayer: 0x55e4d1087400 [3/9457c621-e2d5-43c5-bb20-fb14f55cf969] start: already running 2021-03-09T11:44:20.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.938+0000 7f0441bcd700 -1 rbd::mirror::ImageReplayer: 0x55e4d1086c80 [3/978a796b-0fd5-40e0-ba5f-e4a4399f10d1] start: already running 2021-03-09T11:44:20.943 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.938+0000 7f04423ce700 -1 rbd::mirror::ImageReplayer: 0x55e4d1087180 [3/9d177d94-7bc5-46be-8fff-f48edf3c1f9d] start: already running 2021-03-09T11:44:20.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.938+0000 7f0441bcd700 -1 rbd::mirror::ImageReplayer: 0x55e4d1087b80 [3/aec0a6af-f0ca-401c-a5d1-8f250674262a] start: already running 2021-03-09T11:44:20.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.938+0000 7f04423ce700 -1 rbd::mirror::ImageReplayer: 0x55e4d1087900 [3/b1eacee4-665d-41b5-9f60-43674f116887] start: already running 2021-03-09T11:44:20.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.938+0000 7f0441bcd700 -1 rbd::mirror::ImageReplayer: 0x55e4d1087e00 [3/b2b08618-0c38-4fe7-92b1-937ad9c546bf] start: already running 2021-03-09T11:44:20.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T11:44:20.938+0000 7f04423ce700 -1 rbd::mirror::ImageReplayer: 0x55e4d1088300 [3/bd4ef0c3-a3a7-42f2-8034-c828cab83717] start: already running 2021-03-09T11:44:23.849 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T11:44:23.850 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:44:23.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:44:23.850 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T11:44:23.851 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T11:44:23.851 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T11:44:23.854 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T11:44:23.854 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T11:45:45.855 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T11:45:45.857 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T11:45:45.857 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T11:45:45.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T11:45:45.852+0000 7f7aa1c3e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 77707) UID: 1000 2021-03-09T11:45:46.058 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:45:50.858 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T11:45:50.858 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:45:50.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:45:50.859 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T11:45:50.859 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T11:45:50.860 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T11:45:50.862 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T11:45:50.863 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T11:46:07.761 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T11:46:07.762 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T11:46:07.763 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T11:46:07.764 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:46:07.760+0000 7f1af6cf2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 77164) UID: 1000 2021-03-09T11:46:07.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:46:10.764 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T11:46:10.765 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:46:10.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:46:10.766 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T11:46:10.766 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T11:46:10.767 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T11:46:10.770 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T11:46:10.770 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T11:46:40.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:46:40.091+0000 7fca4e10f700 -1 rbd::mirror::ImageReplayer: 0x55ea5559ef80 [3/18a549b3-61f2-46e5-af4d-1a9b02c6e0ad] start: already running 2021-03-09T11:47:39.864 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9830103097829985) > weight (0.25) 2021-03-09T11:47:39.868 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7431816596541757) > weight (0.25) 2021-03-09T11:47:39.868 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.30637207421481016) > weight (0.25) 2021-03-09T11:47:39.868 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8894817388909599) > weight (0.25) 2021-03-09T11:47:39.868 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T11:47:43.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5403823899886234) > weight (0.25) 2021-03-09T11:47:43.771 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4885655389582301) > weight (0.25) 2021-03-09T11:47:43.772 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4670447420957209) > weight (0.25) 2021-03-09T11:47:43.772 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2874234579215662) > weight (0.25) 2021-03-09T11:47:43.772 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T11:49:10.774 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32273460975887047) > weight (0.25) 2021-03-09T11:49:10.776 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6659839731647674) > weight (0.25) 2021-03-09T11:49:10.776 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.916463875444796) > weight (0.25) 2021-03-09T11:49:10.776 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7004906988419167) > weight (0.25) 2021-03-09T11:49:10.777 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-03-09T11:49:23.869 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2999955547850871) > weight (0.25) 2021-03-09T11:49:23.870 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.937173992071704) > weight (0.25) 2021-03-09T11:49:23.871 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T11:49:23.872 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T11:49:23.873 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T11:49:23.874 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T11:49:23.869+0000 7f41b74b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 76682) UID: 1000 2021-03-09T11:49:24.074 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:49:29.874 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T11:49:29.875 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:49:29.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:49:29.876 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T11:49:29.876 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T11:49:29.877 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T11:49:29.879 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T11:49:29.880 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T11:50:21.777 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T11:50:21.779 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T11:50:21.779 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T11:50:21.780 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:50:21.776+0000 7fca4e910700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 77900) UID: 1000 2021-03-09T11:50:21.981 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:50:27.781 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T11:50:27.781 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:50:27.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:50:27.783 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T11:50:27.783 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T11:50:27.784 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T11:50:27.787 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T11:50:27.787 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T11:50:56.881 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6636997877355757) > weight (0.25) 2021-03-09T11:50:56.883 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T11:50:56.883 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T11:50:56.884 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T11:50:56.884 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:50:56.879+0000 7fa35eace700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 77457) UID: 1000 2021-03-09T11:50:57.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:51:04.885 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T11:51:04.887 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:51:04.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:51:04.888 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T11:51:04.888 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T11:51:04.889 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T11:51:04.895 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T11:51:04.896 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T11:51:32.788 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9200009878563763) > weight (0.25) 2021-03-09T11:51:32.790 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4486963826918914) > weight (0.25) 2021-03-09T11:51:32.790 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6641861375744539) > weight (0.25) 2021-03-09T11:51:32.790 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9193580419805775) > weight (0.25) 2021-03-09T11:51:32.790 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T11:52:10.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42261413179991525) > weight (0.25) 2021-03-09T11:52:10.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9131079182370964) > weight (0.25) 2021-03-09T11:52:10.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9099214690036552) > weight (0.25) 2021-03-09T11:52:10.901 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5997474221027491) > weight (0.25) 2021-03-09T11:52:10.901 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T11:52:41.792 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9973544786741357) > weight (0.25) 2021-03-09T11:52:41.793 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.45068806926019467) > weight (0.25) 2021-03-09T11:52:41.793 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4816612720287211) > weight (0.25) 2021-03-09T11:52:41.794 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7791294261039969) > weight (0.25) 2021-03-09T11:52:41.794 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T11:53:54.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8218230798348268) > weight (0.25) 2021-03-09T11:53:54.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9201730524632955) > weight (0.25) 2021-03-09T11:53:54.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9092130468203087) > weight (0.25) 2021-03-09T11:53:54.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6106018795114728) > weight (0.25) 2021-03-09T11:53:54.906 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T11:54:22.795 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T11:54:22.798 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T11:54:22.798 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T11:54:22.799 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:54:22.796+0000 7f6dee3ad700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 78094) UID: 1000 2021-03-09T11:54:23.000 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:54:26.799 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T11:54:26.800 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:54:26.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:54:26.801 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T11:54:26.801 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T11:54:26.801 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T11:54:26.804 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T11:54:26.804 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T11:55:47.908 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.491171124325123) > weight (0.25) 2021-03-09T11:55:47.910 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T11:55:47.911 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T11:55:47.912 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T11:55:47.913 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T11:55:47.910+0000 7fd5bba3b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 78193) UID: 1000 2021-03-09T11:55:48.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:55:50.912 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T11:55:50.913 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:55:50.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:55:50.914 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T11:55:50.914 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T11:55:50.915 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T11:55:50.917 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T11:55:50.918 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T11:55:55.805 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T11:55:55.806 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T11:55:55.807 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T11:55:55.807 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T11:55:55.802+0000 7f3324991700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 78287) UID: 1000 2021-03-09T11:55:56.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:55:57.807 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T11:55:57.819 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:55:57.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:55:57.820 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T11:55:57.820 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T11:55:57.821 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T11:55:57.824 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T11:55:57.824 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T11:57:27.919 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3680676059385225) > weight (0.25) 2021-03-09T11:57:27.922 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.519631888459421) > weight (0.25) 2021-03-09T11:57:27.922 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5565244459018432) > weight (0.25) 2021-03-09T11:57:27.923 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T11:57:27.924 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T11:57:27.924 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T11:57:27.925 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T11:57:27.920+0000 7f4f624d6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 74870) UID: 1000 2021-03-09T11:57:28.125 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:57:28.925 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T11:57:28.925 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:57:28.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:57:28.926 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T11:57:28.926 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T11:57:28.927 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T11:57:28.929 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T11:57:28.930 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-03-09T11:57:38.825 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7656921386664335) > weight (0.25) 2021-03-09T11:57:38.825 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8206283285270579) > weight (0.25) 2021-03-09T11:57:38.825 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4964732099954311) > weight (0.25) 2021-03-09T11:57:38.826 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T11:57:38.826 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T11:57:38.826 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T11:57:38.827 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T11:57:38.824+0000 7f15347d2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 76974) UID: 1000 2021-03-09T11:57:39.028 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:57:42.827 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T11:57:42.827 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:57:42.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:57:42.828 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T11:57:42.829 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T11:57:42.829 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T11:57:42.832 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T11:57:42.832 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-03-09T11:59:08.833 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.398467838639721) > weight (0.25) 2021-03-09T11:59:08.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6841173268953129) > weight (0.25) 2021-03-09T11:59:08.876 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7645498084799109) > weight (0.25) 2021-03-09T11:59:08.877 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9848547672377632) > weight (0.25) 2021-03-09T11:59:08.877 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T11:59:26.931 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4841586586618445) > weight (0.25) 2021-03-09T11:59:26.937 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4335302003796727) > weight (0.25) 2021-03-09T11:59:26.938 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T11:59:26.939 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T11:59:26.940 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T11:59:26.941 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T11:59:26.934+0000 7fb2c8a52700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 78003) UID: 1000 2021-03-09T11:59:27.140 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T11:59:30.941 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T11:59:30.943 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T11:59:30.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T11:59:30.944 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T11:59:30.944 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T11:59:30.944 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T11:59:30.947 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T11:59:30.948 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T12:00:00.148 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:00:00.142+0000 7fbf128b2700 -1 rbd::mirror::ImageReplayer: 0x5624e87d3b80 [3/76a412aa-913f-4f54-ad5f-0dbad3c607cd] start: already running 2021-03-09T12:00:00.149 INFO:tasks.ceph.cluster2.osd.0.smithi144.stderr:2021-03-09T12:00:00.142+0000 7f0e42124700 -1 reset not still connected to 0x5573fc05a490 2021-03-09T12:00:00.150 INFO:tasks.ceph.cluster2.osd.0.smithi144.stderr:2021-03-09T12:00:00.142+0000 7f0e42124700 -1 reset not still connected to 0x5573fd0a15f0 2021-03-09T12:00:37.879 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.34386608848595457) > weight (0.25) 2021-03-09T12:00:37.882 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T12:00:37.884 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T12:00:37.885 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T12:00:37.887 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T12:00:37.881+0000 7f0442bcf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 77264) UID: 1000 2021-03-09T12:00:38.286 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:00:46.887 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T12:00:46.888 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:00:46.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:00:46.889 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T12:00:46.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T12:00:46.890 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T12:00:46.893 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T12:00:46.893 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T12:01:07.948 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6382686712534833) > weight (0.25) 2021-03-09T12:01:08.667 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.592110225318054) > weight (0.25) 2021-03-09T12:01:08.668 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3632830956964823) > weight (0.25) 2021-03-09T12:01:08.668 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T12:01:08.669 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T12:01:08.669 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T12:01:08.670 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T12:01:08.669+0000 7f5e44394700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 78596) UID: 1000 2021-03-09T12:01:08.871 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:01:09.670 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T12:01:09.671 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:01:09.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:01:09.672 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T12:01:09.672 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T12:01:09.672 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T12:01:09.675 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T12:01:09.675 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T12:02:25.894 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T12:02:25.900 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T12:02:25.900 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T12:02:25.901 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T12:02:25.899+0000 7f187352d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 78480) UID: 1000 2021-03-09T12:02:26.102 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:02:29.901 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T12:02:29.901 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:02:29.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:02:29.902 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T12:02:29.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T12:02:29.903 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T12:02:29.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T12:02:29.907 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T12:03:03.677 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T12:03:03.678 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T12:03:03.678 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T12:03:03.679 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T12:03:03.675+0000 7f498b91d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 77809) UID: 1000 2021-03-09T12:03:03.879 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:03:09.679 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T12:03:09.679 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:03:09.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:03:09.680 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T12:03:09.680 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T12:03:09.681 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T12:03:09.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T12:03:09.684 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T12:03:43.908 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8324404409451184) > weight (0.25) 2021-03-09T12:03:43.909 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T12:03:43.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T12:03:43.910 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T12:03:43.910 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T12:03:43.906+0000 7ff6cd25d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 78880) UID: 1000 2021-03-09T12:03:44.111 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:03:44.910 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T12:03:44.911 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:03:44.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:03:44.912 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T12:03:44.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T12:03:44.912 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T12:03:44.915 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T12:03:44.915 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T12:04:21.685 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7464049906712928) > weight (0.25) 2021-03-09T12:04:21.687 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T12:04:21.688 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T12:04:21.689 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T12:04:21.689 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T12:04:21.690 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:04:21.691 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T12:04:21.681+0000 7ff9a60f6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 78389) UID: 1000 2021-03-09T12:04:22.090 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:04:27.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:04:27.793 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T12:04:27.793 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T12:04:27.793 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T12:04:27.797 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T12:04:27.798 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T12:05:04.916 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T12:05:04.923 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T12:05:04.924 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T12:05:04.924 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T12:05:04.925 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:05:04.925 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T12:05:04.917+0000 7fe98bd1f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 79071) UID: 1000 2021-03-09T12:05:05.125 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:05:11.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:05:11.027 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T12:05:11.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T12:05:11.027 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T12:05:11.030 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T12:05:11.030 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T12:06:09.918 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:06:09.914+0000 7f901a51c700 -1 reset not still connected to 0x5588ffe7cf20 2021-03-09T12:06:09.919 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:06:09.914+0000 7f901a51c700 -1 reset not still connected to 0x55890173b2b0 2021-03-09T12:06:22.799 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29444415496907994) > weight (0.25) 2021-03-09T12:06:22.800 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9591112833428503) > weight (0.25) 2021-03-09T12:06:22.800 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6282186654291071) > weight (0.25) 2021-03-09T12:06:22.801 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T12:06:22.802 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T12:06:22.802 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T12:06:22.805 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T12:06:22.799+0000 7f19b887e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 78979) UID: 1000 2021-03-09T12:06:23.005 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:06:27.804 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T12:06:27.804 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:06:27.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:06:27.805 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T12:06:27.805 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T12:06:27.806 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T12:06:27.808 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T12:06:27.809 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T12:06:40.031 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6876191576416819) > weight (0.25) 2021-03-09T12:06:40.032 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5652361717228255) > weight (0.25) 2021-03-09T12:06:40.032 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3185437051868667) > weight (0.25) 2021-03-09T12:06:40.032 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T12:06:40.033 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T12:06:40.033 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T12:06:40.034 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T12:06:40.031+0000 7f01db9e1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 78687) UID: 1000 2021-03-09T12:06:40.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:06:46.034 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T12:06:46.035 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:06:46.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:06:46.035 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T12:06:46.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T12:06:46.036 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T12:06:46.039 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T12:06:46.039 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T12:08:08.810 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T12:08:08.812 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T12:08:08.812 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T12:08:08.813 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T12:08:08.810+0000 7f9c24137700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 79170) UID: 1000 2021-03-09T12:08:09.014 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:08:14.040 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8120212075973032) > weight (0.25) 2021-03-09T12:08:14.040 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.40433269816322515) > weight (0.25) 2021-03-09T12:08:14.041 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T12:08:14.041 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T12:08:14.041 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T12:08:14.042 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T12:08:14.042 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:08:14.044 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T12:08:14.037+0000 7fdf990bf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 77615) UID: 1000 2021-03-09T12:08:14.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:08:15.590 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T12:08:15.585+0000 7f082a074700 -1 reset not still connected to 0x55a9bb2321a0 2021-03-09T12:08:15.592 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T12:08:15.585+0000 7f082a074700 -1 reset not still connected to 0x55a9bb232dd0 2021-03-09T12:08:16.813 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T12:08:16.813 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:08:16.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:08:16.814 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T12:08:16.814 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T12:08:16.815 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T12:08:16.818 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T12:08:16.818 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T12:08:20.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:08:20.144 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T12:08:20.144 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T12:08:20.144 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T12:08:20.147 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T12:08:20.147 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T12:09:53.819 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5791872759926523) > weight (0.25) 2021-03-09T12:09:53.822 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7952420884229965) > weight (0.25) 2021-03-09T12:09:53.823 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T12:09:53.823 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T12:09:53.824 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T12:09:53.826 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:09:53.824+0000 7fbf130b3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 78788) UID: 1000 2021-03-09T12:09:54.026 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:09:55.825 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T12:09:55.825 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:09:55.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:09:55.826 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T12:09:55.826 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T12:09:55.827 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T12:09:55.829 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T12:09:55.830 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T12:10:10.149 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4061758329928148) > weight (0.25) 2021-03-09T12:10:10.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.36349447045065075) > weight (0.25) 2021-03-09T12:10:10.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7853355701384881) > weight (0.25) 2021-03-09T12:10:10.152 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6415847289170322) > weight (0.25) 2021-03-09T12:10:10.152 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T12:11:43.831 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5073674971013666) > weight (0.25) 2021-03-09T12:11:43.834 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7195400067847316) > weight (0.25) 2021-03-09T12:11:43.834 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T12:11:43.836 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T12:11:43.836 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T12:11:43.838 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:11:43.834+0000 7fb4e45a9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 79938) UID: 1000 2021-03-09T12:11:44.038 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:11:48.838 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T12:11:48.847 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:11:48.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:11:48.848 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T12:11:48.848 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T12:11:48.848 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T12:11:48.851 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T12:11:48.852 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T12:12:04.154 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.850263488195858) > weight (0.25) 2021-03-09T12:12:04.155 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T12:12:04.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T12:12:04.156 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T12:12:04.156 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T12:12:04.150+0000 7f62ca38b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 79261) UID: 1000 2021-03-09T12:12:04.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:12:09.157 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T12:12:09.157 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:12:09.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:12:09.158 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T12:12:09.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T12:12:09.159 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T12:12:09.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T12:12:09.162 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-03-09T12:13:30.163 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7529239258795705) > weight (0.25) 2021-03-09T12:13:30.164 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7056146227629599) > weight (0.25) 2021-03-09T12:13:30.164 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.42990165924030943) > weight (0.25) 2021-03-09T12:13:30.164 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5245824290826445) > weight (0.25) 2021-03-09T12:13:30.164 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T12:13:37.853 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7937703671267529) > weight (0.25) 2021-03-09T12:13:37.853 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T12:13:37.854 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T12:13:37.854 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T12:13:37.855 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T12:13:37.852+0000 7fe53117c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 79362) UID: 1000 2021-03-09T12:13:38.056 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:13:42.855 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T12:13:42.856 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:13:42.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:13:42.857 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T12:13:42.857 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T12:13:42.857 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T12:13:42.860 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T12:13:42.861 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T12:14:40.166 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.27492947256742983) > weight (0.25) 2021-03-09T12:14:40.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.49713869482043305) > weight (0.25) 2021-03-09T12:14:40.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5487749141561532) > weight (0.25) 2021-03-09T12:14:40.169 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.38139322458420966) > weight (0.25) 2021-03-09T12:14:40.169 INFO:tasks.rbd_mirror_thrash:waiting for 81 secs before thrashing 2021-03-09T12:14:54.862 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T12:14:54.865 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T12:14:54.865 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T12:14:54.866 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T12:14:54.863+0000 7fd2cbfb8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 79747) UID: 1000 2021-03-09T12:14:55.066 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:15:02.866 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T12:15:02.876 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:15:02.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:15:02.876 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T12:15:02.877 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T12:15:02.878 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T12:15:02.881 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T12:15:02.881 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T12:16:01.171 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5675216694298953) > weight (0.25) 2021-03-09T12:16:01.173 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T12:16:01.174 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T12:16:01.175 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T12:16:01.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T12:16:01.170+0000 7f45c601b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 80122) UID: 1000 2021-03-09T12:16:01.376 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:16:04.175 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T12:16:04.176 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:16:04.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:16:04.177 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T12:16:04.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T12:16:04.178 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T12:16:04.181 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T12:16:04.181 INFO:tasks.rbd_mirror_thrash:waiting for 86 secs before thrashing 2021-03-09T12:16:11.882 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3350932306487937) > weight (0.25) 2021-03-09T12:16:11.885 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7800922832731069) > weight (0.25) 2021-03-09T12:16:11.885 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T12:16:11.885 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T12:16:11.886 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T12:16:11.887 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:16:11.881+0000 7f4cba19a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 80031) UID: 1000 2021-03-09T12:16:12.087 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:16:18.886 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T12:16:18.889 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:16:18.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:16:18.890 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T12:16:18.890 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T12:16:18.891 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T12:16:18.893 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T12:16:18.893 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T12:16:32.648 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T12:16:32.637+0000 7f082a074700 -1 reset not still connected to 0x55aa12b21400 2021-03-09T12:17:30.182 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2745785732985756) > weight (0.25) 2021-03-09T12:17:30.185 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.35167712064019696) > weight (0.25) 2021-03-09T12:17:30.185 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.32162963571027414) > weight (0.25) 2021-03-09T12:17:30.185 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2723489723543647) > weight (0.25) 2021-03-09T12:17:30.185 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T12:17:49.894 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5332291278567215) > weight (0.25) 2021-03-09T12:17:49.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.474223302487512) > weight (0.25) 2021-03-09T12:17:49.897 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.531134741496803) > weight (0.25) 2021-03-09T12:17:49.897 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T12:17:49.898 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T12:17:49.899 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T12:17:49.900 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T12:17:49.896+0000 7f3af318e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 79556) UID: 1000 2021-03-09T12:17:50.100 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:17:58.899 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T12:17:58.903 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:17:58.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:17:58.904 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T12:17:58.904 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T12:17:58.904 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T12:17:58.907 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T12:17:58.908 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T12:19:21.186 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T12:19:21.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T12:19:21.191 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T12:19:21.192 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T12:19:21.186+0000 7f06b10e2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 79456) UID: 1000 2021-03-09T12:19:21.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:19:25.192 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T12:19:25.193 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:19:25.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:19:25.194 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T12:19:25.194 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T12:19:25.194 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T12:19:25.199 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T12:19:25.199 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T12:19:43.908 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8391591651471801) > weight (0.25) 2021-03-09T12:19:43.911 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9206919276749118) > weight (0.25) 2021-03-09T12:19:43.911 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3872236719841081) > weight (0.25) 2021-03-09T12:19:43.911 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8453905375453726) > weight (0.25) 2021-03-09T12:19:43.911 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T12:21:21.200 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9993743226358069) > weight (0.25) 2021-03-09T12:21:21.202 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.922549669943263) > weight (0.25) 2021-03-09T12:21:21.202 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T12:21:21.204 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T12:21:21.204 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T12:21:21.205 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T12:21:21.200+0000 7fddc8b20700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 79838) UID: 1000 2021-03-09T12:21:21.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:21:23.205 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T12:21:23.205 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:21:23.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:21:23.206 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T12:21:23.207 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T12:21:23.207 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T12:21:23.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T12:21:23.210 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T12:21:42.913 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T12:21:42.917 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T12:21:42.917 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T12:21:42.918 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T12:21:42.916+0000 7febc6b5e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80317) UID: 1000 2021-03-09T12:21:43.118 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:21:44.082 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T12:21:44.080+0000 7fb7177a6700 -1 rbd::mirror::ImageReplayer: 0x55ac5ae39e00 [3/cbb866ff-7e4b-4de9-80d6-4f875228d852] start: already running 2021-03-09T12:21:44.084 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:21:44.080+0000 7f546b8f2700 -1 rbd::mirror::ImageReplayer: 0x55926e1dbe00 [3/76a412aa-913f-4f54-ad5f-0dbad3c607cd] start: already running 2021-03-09T12:21:44.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:21:44.080+0000 7f546b8f2700 -1 rbd::mirror::ImageReplayer: 0x55926e1dc080 [3/7c5811a1-6f19-4292-9a3a-335262621df8] start: already running 2021-03-09T12:21:44.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:21:44.080+0000 7f546b8f2700 -1 rbd::mirror::ImageReplayer: 0x55926e1dc300 [3/7cda0e60-383d-4c8c-85dd-263c9aac913a] start: already running 2021-03-09T12:21:44.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:21:44.080+0000 7f546b8f2700 -1 rbd::mirror::ImageReplayer: 0x55926e1db900 [3/8853c04a-4df4-47fb-9df9-6cebf19cc680] start: already running 2021-03-09T12:21:44.085 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:21:44.080+0000 7f546b8f2700 -1 rbd::mirror::ImageReplayer: 0x55926e1dbb80 [3/9d177d94-7bc5-46be-8fff-f48edf3c1f9d] start: already running 2021-03-09T12:21:44.086 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:21:44.080+0000 7f546b8f2700 -1 rbd::mirror::ImageReplayer: 0x55926e1dac80 [3/dc036c7f-f3e2-4032-b379-9d4600fb05ba] start: already running 2021-03-09T12:21:44.086 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:21:44.080+0000 7f546b8f2700 -1 rbd::mirror::ImageReplayer: 0x55926e1dc580 [3/f5ac9111-3dda-4ae3-aa2c-fb3fde1f88c5] start: already running 2021-03-09T12:21:44.086 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:21:44.080+0000 7f546b8f2700 -1 rbd::mirror::ImageReplayer: 0x55926e1daf00 [3/fcee6884-fbbf-4b09-bfba-340ddeb5e81b] start: already running 2021-03-09T12:21:49.918 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T12:21:49.918 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:21:49.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:21:49.919 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T12:21:49.920 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T12:21:49.920 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T12:21:49.923 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T12:21:49.923 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T12:23:16.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5410208917060335) > weight (0.25) 2021-03-09T12:23:16.214 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9748563913641441) > weight (0.25) 2021-03-09T12:23:16.215 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3859523876405252) > weight (0.25) 2021-03-09T12:23:16.215 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.33943067116098136) > weight (0.25) 2021-03-09T12:23:16.215 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T12:23:29.119 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T12:23:29.114+0000 7fc0e0d45700 -1 reset not still connected to 0x55e8d1642f70 2021-03-09T12:23:40.924 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.45283232011931596) > weight (0.25) 2021-03-09T12:23:40.925 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5296528794344103) > weight (0.25) 2021-03-09T12:23:40.925 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T12:23:40.926 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T12:23:40.926 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T12:23:40.927 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:23:40.922+0000 7f546c0f3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 80507) UID: 1000 2021-03-09T12:23:41.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:23:41.927 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T12:23:41.927 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:23:41.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:23:41.928 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T12:23:41.928 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T12:23:41.929 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T12:23:41.932 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T12:23:41.932 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T12:24:50.216 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8542234374641625) > weight (0.25) 2021-03-09T12:24:50.217 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3119023168560938) > weight (0.25) 2021-03-09T12:24:50.217 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T12:24:50.218 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T12:24:50.218 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T12:24:50.219 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T12:24:50.217+0000 7f6f52dea700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 80796) UID: 1000 2021-03-09T12:24:50.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:24:51.933 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9961367711266411) > weight (0.25) 2021-03-09T12:24:51.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7437998268779915) > weight (0.25) 2021-03-09T12:24:51.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8308153394782187) > weight (0.25) 2021-03-09T12:24:51.934 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8885555188630071) > weight (0.25) 2021-03-09T12:24:51.934 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T12:24:53.220 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T12:24:53.220 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:24:53.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:24:53.221 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T12:24:53.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T12:24:53.222 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T12:24:53.225 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T12:24:53.225 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T12:26:34.227 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5728415043446615) > weight (0.25) 2021-03-09T12:26:34.230 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4525159311918332) > weight (0.25) 2021-03-09T12:26:34.230 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8526683684198518) > weight (0.25) 2021-03-09T12:26:34.231 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.578678543185067) > weight (0.25) 2021-03-09T12:26:34.231 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T12:26:35.935 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5518341700994207) > weight (0.25) 2021-03-09T12:26:35.936 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4790047178938749) > weight (0.25) 2021-03-09T12:26:35.936 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.73564049003915) > weight (0.25) 2021-03-09T12:26:35.936 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8267281104584124) > weight (0.25) 2021-03-09T12:26:35.936 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T12:28:12.937 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T12:28:12.958 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T12:28:12.958 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T12:28:12.959 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T12:28:12.953+0000 7f3de4720700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 80895) UID: 1000 2021-03-09T12:28:13.159 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:28:17.231 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8130691724681889) > weight (0.25) 2021-03-09T12:28:17.232 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.31729682623720756) > weight (0.25) 2021-03-09T12:28:17.232 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.32964338209086985) > weight (0.25) 2021-03-09T12:28:17.232 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T12:28:17.233 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T12:28:17.233 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T12:28:17.234 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T12:28:17.229+0000 7fbb4193f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 79647) UID: 1000 2021-03-09T12:28:17.635 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:28:18.959 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T12:28:18.960 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:28:18.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:28:18.961 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T12:28:18.961 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T12:28:18.961 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T12:28:18.965 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T12:28:18.965 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T12:28:19.234 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T12:28:19.234 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:28:19.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:28:19.236 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T12:28:19.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T12:28:19.237 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T12:28:19.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T12:28:19.240 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T12:30:09.242 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.624227423842899) > weight (0.25) 2021-03-09T12:30:09.245 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3676689352724667) > weight (0.25) 2021-03-09T12:30:09.246 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9668660519252935) > weight (0.25) 2021-03-09T12:30:09.246 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.2674116596675856) > weight (0.25) 2021-03-09T12:30:09.246 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T12:30:11.966 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.659809550530961) > weight (0.25) 2021-03-09T12:30:11.966 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T12:30:11.968 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T12:30:11.968 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T12:30:11.969 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T12:30:11.963+0000 7fa7e03e8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 80222) UID: 1000 2021-03-09T12:30:12.169 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:30:19.969 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T12:30:20.038 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:30:20.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:30:20.039 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T12:30:20.039 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T12:30:20.040 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T12:30:20.047 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T12:30:20.047 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T12:30:40.294 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T12:30:40.291+0000 7f082a074700 -1 reset not still connected to 0x55a99c43a630 2021-03-09T12:30:40.295 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T12:30:40.291+0000 7f082a074700 -1 reset not still connected to 0x55a99f7e41a0 2021-03-09T12:30:40.295 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T12:30:40.291+0000 7f082a074700 -1 reset not still connected to 0x55a9a25c2080 2021-03-09T12:31:12.247 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T12:31:12.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T12:31:12.250 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T12:31:12.251 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T12:31:12.246+0000 7f13f87cc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 80695) UID: 1000 2021-03-09T12:31:12.452 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:31:19.251 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T12:31:19.252 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:31:19.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:31:19.253 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T12:31:19.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T12:31:19.254 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T12:31:19.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T12:31:19.257 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T12:31:57.048 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9771197457102888) > weight (0.25) 2021-03-09T12:31:57.052 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3440690941738709) > weight (0.25) 2021-03-09T12:31:57.052 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9936029968865169) > weight (0.25) 2021-03-09T12:31:57.052 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4580327973158792) > weight (0.25) 2021-03-09T12:31:57.052 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T12:32:57.054 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6912820563191554) > weight (0.25) 2021-03-09T12:32:57.095 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T12:32:57.096 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T12:32:57.097 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T12:32:57.098 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T12:32:57.092+0000 7faacbeb5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 81376) UID: 1000 2021-03-09T12:32:57.298 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:33:03.098 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T12:33:03.100 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:33:03.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:33:03.101 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T12:33:03.101 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T12:33:03.105 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T12:33:03.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T12:33:03.108 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T12:33:13.258 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5383954821837281) > weight (0.25) 2021-03-09T12:33:13.259 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T12:33:13.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T12:33:13.260 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T12:33:13.260 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T12:33:13.256+0000 7f80b357d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 80408) UID: 1000 2021-03-09T12:33:13.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:33:21.261 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T12:33:21.262 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:33:21.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:33:21.263 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T12:33:21.264 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T12:33:21.264 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T12:33:21.267 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T12:33:21.267 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T12:34:06.110 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T12:34:06.114 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T12:34:06.114 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T12:34:06.115 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T12:34:06.111+0000 7f8316701700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 81184) UID: 1000 2021-03-09T12:34:06.315 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:34:10.115 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T12:34:10.116 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:34:10.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:34:10.117 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T12:34:10.117 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T12:34:10.117 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T12:34:10.120 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T12:34:10.120 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T12:34:21.269 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.40844566757936507) > weight (0.25) 2021-03-09T12:34:21.269 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.36683734292361214) > weight (0.25) 2021-03-09T12:34:21.269 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6422970951300446) > weight (0.25) 2021-03-09T12:34:21.270 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4337562852629012) > weight (0.25) 2021-03-09T12:34:21.270 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T12:35:10.122 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T12:35:10.127 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T12:35:10.128 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T12:35:10.129 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T12:35:10.122+0000 7f244c16d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 81760) UID: 1000 2021-03-09T12:35:10.329 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:35:12.128 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T12:35:12.129 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:35:12.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:35:12.130 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T12:35:12.130 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T12:35:12.130 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T12:35:12.134 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T12:35:12.135 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T12:35:35.271 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7045897846883792) > weight (0.25) 2021-03-09T12:35:35.274 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T12:35:35.275 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T12:35:35.275 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T12:35:35.275 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T12:35:35.275 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:35:35.278 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T12:35:35.274+0000 7f346dc0f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 81660) UID: 1000 2021-03-09T12:35:35.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:35:41.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:35:41.377 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T12:35:41.378 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T12:35:41.378 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T12:35:41.382 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T12:35:41.383 INFO:tasks.rbd_mirror_thrash:waiting for 80 secs before thrashing 2021-03-09T12:36:41.136 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T12:36:41.140 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T12:36:41.140 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T12:36:41.141 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T12:36:41.136+0000 7fa965709700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 81855) UID: 1000 2021-03-09T12:36:41.341 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:36:42.184 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:36:42.177+0000 7f901a51c700 -1 reset not still connected to 0x558901a688a0 2021-03-09T12:36:48.141 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T12:36:48.143 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:36:48.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:36:48.144 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T12:36:48.144 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T12:36:48.144 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T12:36:48.147 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T12:36:48.148 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T12:37:01.384 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6939306490182241) > weight (0.25) 2021-03-09T12:37:01.385 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2788769259365018) > weight (0.25) 2021-03-09T12:37:01.386 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T12:37:01.386 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T12:37:01.387 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T12:37:01.388 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T12:37:01.384+0000 7f218cc39700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 81080) UID: 1000 2021-03-09T12:37:01.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:37:06.388 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T12:37:06.392 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:37:06.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:37:06.392 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T12:37:06.393 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T12:37:06.393 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T12:37:06.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T12:37:06.396 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T12:37:12.597 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:37:12.588+0000 7f901a51c700 -1 reset not still connected to 0x5588f8a22d80 2021-03-09T12:37:12.597 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:37:12.588+0000 7f901a51c700 -1 reset not still connected to 0x5588f923cea0 2021-03-09T12:37:12.598 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:37:12.588+0000 7f901a51c700 -1 reset not still connected to 0x5588fb9470c0 2021-03-09T12:37:12.600 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:37:12.592+0000 7f901a51c700 -1 reset not still connected to 0x5588fb947810 2021-03-09T12:37:12.600 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:37:12.592+0000 7f901a51c700 -1 reset not still connected to 0x5588ffe7c970 2021-03-09T12:37:12.601 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:37:12.592+0000 7f901a51c700 -1 reset not still connected to 0x558901a66a90 2021-03-09T12:37:33.339 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:37:33.331+0000 7f901a51c700 -1 reset not still connected to 0x558904ab01a0 2021-03-09T12:38:09.397 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7052575543957068) > weight (0.25) 2021-03-09T12:38:09.400 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7584244308680864) > weight (0.25) 2021-03-09T12:38:09.400 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T12:38:09.401 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T12:38:09.401 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T12:38:09.402 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T12:38:09.399+0000 7f8627a36700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 82138) UID: 1000 2021-03-09T12:38:09.603 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:38:11.149 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.818133857157863) > weight (0.25) 2021-03-09T12:38:11.149 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7991697532601546) > weight (0.25) 2021-03-09T12:38:11.150 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T12:38:11.150 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T12:38:11.151 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T12:38:11.151 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:38:11.147+0000 7f74d2e9a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 80988) UID: 1000 2021-03-09T12:38:11.352 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:38:15.151 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T12:38:15.152 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:38:15.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:38:15.153 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T12:38:15.153 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T12:38:15.153 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T12:38:15.156 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T12:38:15.156 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T12:38:17.402 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T12:38:17.407 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:38:17.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:38:17.408 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T12:38:17.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T12:38:17.409 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T12:38:17.419 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T12:38:17.420 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T12:39:15.158 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6806986089903938) > weight (0.25) 2021-03-09T12:39:15.161 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T12:39:15.162 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T12:39:15.163 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T12:39:15.163 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T12:39:15.162+0000 7fd7cca34700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 81569) UID: 1000 2021-03-09T12:39:15.364 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:39:17.164 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T12:39:17.166 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:39:17.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:39:17.167 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T12:39:17.167 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T12:39:17.168 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T12:39:17.180 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T12:39:17.181 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T12:39:33.601 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T12:39:33.596+0000 7fdda4a8d700 -1 reset not still connected to 0x55aefc557380 2021-03-09T12:39:39.794 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:39:39.784+0000 7f901a51c700 -1 reset not still connected to 0x5588f8a22d80 2021-03-09T12:39:39.794 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:39:39.784+0000 7f901a51c700 -1 reset not still connected to 0x5588f923cea0 2021-03-09T12:39:39.795 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:39:39.784+0000 7f901a51c700 -1 reset not still connected to 0x5588fb9470c0 2021-03-09T12:39:39.795 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:39:39.788+0000 7f901a51c700 -1 reset not still connected to 0x5588fb947810 2021-03-09T12:39:39.796 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:39:39.788+0000 7f901a51c700 -1 reset not still connected to 0x5588ffe7c970 2021-03-09T12:39:50.420 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.36867687649714076) > weight (0.25) 2021-03-09T12:39:50.422 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3097583135505082) > weight (0.25) 2021-03-09T12:39:50.423 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6376050467775582) > weight (0.25) 2021-03-09T12:39:50.423 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7088891317248303) > weight (0.25) 2021-03-09T12:39:50.423 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T12:40:21.182 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T12:40:21.185 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T12:40:21.186 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T12:40:21.187 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T12:40:21.181+0000 7f5d54e06700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 82047) UID: 1000 2021-03-09T12:40:21.387 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:40:29.187 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T12:40:29.190 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:40:29.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:40:29.191 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T12:40:29.191 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T12:40:29.193 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T12:40:29.198 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T12:40:29.198 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T12:40:45.379 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:40:45.374+0000 7f901a51c700 -1 reset not still connected to 0x5588fb0bb110 2021-03-09T12:41:35.425 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.39011319573931547) > weight (0.25) 2021-03-09T12:41:35.428 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4660087669258951) > weight (0.25) 2021-03-09T12:41:35.428 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5982676260153106) > weight (0.25) 2021-03-09T12:41:35.428 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6247161384214972) > weight (0.25) 2021-03-09T12:41:35.428 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T12:42:02.200 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.28006133686500134) > weight (0.25) 2021-03-09T12:42:02.201 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T12:42:02.202 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T12:42:02.202 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T12:42:02.203 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T12:42:02.199+0000 7fd4c9ffe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 82428) UID: 1000 2021-03-09T12:42:02.403 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:42:04.203 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T12:42:04.204 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:42:04.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:42:04.205 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T12:42:04.205 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T12:42:04.205 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T12:42:04.208 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T12:42:04.209 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T12:42:39.430 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T12:42:39.431 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T12:42:39.432 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T12:42:39.432 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T12:42:39.426+0000 7f745305d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 81470) UID: 1000 2021-03-09T12:42:39.833 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:42:44.432 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T12:42:44.433 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:42:44.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:42:44.433 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T12:42:44.434 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T12:42:44.434 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T12:42:44.441 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T12:42:44.441 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T12:42:59.435 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:42:59.430+0000 7f901a51c700 -1 reset not still connected to 0x558904ab01a0 2021-03-09T12:43:52.210 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5217455855973057) > weight (0.25) 2021-03-09T12:43:52.239 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6908991270780561) > weight (0.25) 2021-03-09T12:43:52.240 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7747245289932124) > weight (0.25) 2021-03-09T12:43:52.240 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T12:43:52.240 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T12:43:52.241 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T12:43:52.242 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T12:43:52.237+0000 7fb7187a8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 80603) UID: 1000 2021-03-09T12:43:52.442 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:43:59.242 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T12:43:59.243 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:43:59.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:43:59.244 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T12:43:59.245 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T12:43:59.245 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T12:43:59.255 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T12:43:59.256 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T12:44:15.442 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7752150904483438) > weight (0.25) 2021-03-09T12:44:15.443 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3025309018183443) > weight (0.25) 2021-03-09T12:44:15.444 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T12:44:15.445 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T12:44:15.445 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T12:44:15.446 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T12:44:15.446 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:44:15.447 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T12:44:15.441+0000 7f3f20005700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 82329) UID: 1000 2021-03-09T12:44:15.646 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:44:21.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:44:21.548 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T12:44:21.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T12:44:21.549 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T12:44:21.562 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T12:44:21.563 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T12:45:06.257 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6662153921881924) > weight (0.25) 2021-03-09T12:45:06.257 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8354893891603914) > weight (0.25) 2021-03-09T12:45:06.258 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T12:45:06.258 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T12:45:06.259 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T12:45:06.259 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:45:06.256+0000 7fb0001a6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 82238) UID: 1000 2021-03-09T12:45:06.460 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:45:13.260 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T12:45:13.260 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:45:13.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:45:13.261 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T12:45:13.261 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T12:45:13.262 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T12:45:13.264 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T12:45:13.264 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T12:45:58.564 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4715075074856456) > weight (0.25) 2021-03-09T12:45:58.566 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T12:45:58.567 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T12:45:58.568 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T12:45:58.570 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T12:45:58.563+0000 7f502e34f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 81946) UID: 1000 2021-03-09T12:45:58.769 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:45:59.570 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T12:45:59.570 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:45:59.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:45:59.571 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T12:45:59.571 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T12:45:59.572 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T12:45:59.574 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T12:45:59.575 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T12:46:23.265 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T12:46:23.292 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T12:46:23.293 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T12:46:23.293 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T12:46:23.291+0000 7f9a04efc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 82521) UID: 1000 2021-03-09T12:46:23.494 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:46:30.294 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T12:46:30.298 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:46:30.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:46:30.301 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T12:46:30.302 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T12:46:30.303 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T12:46:30.344 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T12:46:30.346 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T12:47:32.577 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.533075115807623) > weight (0.25) 2021-03-09T12:47:32.619 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5633254740638789) > weight (0.25) 2021-03-09T12:47:32.620 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.48396308259512366) > weight (0.25) 2021-03-09T12:47:32.620 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T12:47:32.621 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T12:47:32.624 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T12:47:32.625 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T12:47:32.617+0000 7fe38f3b9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 81241) UID: 1000 2021-03-09T12:47:33.023 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:47:33.860 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:47:33.846+0000 7f901a51c700 -1 reset not still connected to 0x5588fdfc2970 2021-03-09T12:47:33.861 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:47:33.850+0000 7f901a51c700 -1 reset not still connected to 0x5588fb947330 2021-03-09T12:47:33.864 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:47:33.854+0000 7f901a51c700 -1 reset not still connected to 0x5588fdfc2970 2021-03-09T12:47:39.625 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T12:47:39.627 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:47:39.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:47:39.628 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T12:47:39.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T12:47:39.629 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T12:47:39.651 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T12:47:39.651 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T12:47:54.347 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8915166786738776) > weight (0.25) 2021-03-09T12:47:54.349 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7449735361859304) > weight (0.25) 2021-03-09T12:47:54.349 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.46752023233732254) > weight (0.25) 2021-03-09T12:47:54.350 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T12:47:54.350 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T12:47:54.351 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T12:47:54.351 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T12:47:54.345+0000 7fdbf8c30700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 82807) UID: 1000 2021-03-09T12:47:54.552 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:47:56.352 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T12:47:56.352 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:47:56.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:47:56.353 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T12:47:56.353 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T12:47:56.354 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T12:47:56.356 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T12:47:56.356 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T12:48:45.652 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T12:48:45.655 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T12:48:45.656 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T12:48:45.656 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T12:48:45.652+0000 7f604c62a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 82706) UID: 1000 2021-03-09T12:48:45.857 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:48:52.657 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T12:48:52.657 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:48:52.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:48:52.658 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T12:48:52.658 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T12:48:52.659 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T12:48:52.662 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T12:48:52.662 INFO:tasks.rbd_mirror_thrash:waiting for 99 secs before thrashing 2021-03-09T12:49:33.358 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6805047486056547) > weight (0.25) 2021-03-09T12:49:33.361 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4819866356511677) > weight (0.25) 2021-03-09T12:49:33.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8400124305440116) > weight (0.25) 2021-03-09T12:49:33.362 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3994233746030381) > weight (0.25) 2021-03-09T12:49:33.362 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T12:50:31.663 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5618071105601808) > weight (0.25) 2021-03-09T12:50:31.666 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8932601043195643) > weight (0.25) 2021-03-09T12:50:31.666 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7191813534511489) > weight (0.25) 2021-03-09T12:50:31.666 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T12:50:31.667 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T12:50:31.668 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T12:50:31.671 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T12:50:31.666+0000 7f5693062700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 83283) UID: 1000 2021-03-09T12:50:31.871 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:50:40.669 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T12:50:40.669 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:50:40.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:50:40.670 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T12:50:40.670 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T12:50:40.671 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T12:50:40.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T12:50:40.675 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T12:50:43.498 INFO:tasks.ceph.cluster2.osd.1.smithi144.stderr:2021-03-09T12:50:43.490+0000 7fc0e0d45700 -1 reset not still connected to 0x55e8df74bad0 2021-03-09T12:51:28.363 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2796442178531783) > weight (0.25) 2021-03-09T12:51:28.385 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5543423052280364) > weight (0.25) 2021-03-09T12:51:28.385 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6222149569488641) > weight (0.25) 2021-03-09T12:51:28.385 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4645050687171942) > weight (0.25) 2021-03-09T12:51:28.386 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T12:51:42.059 INFO:tasks.ceph.cluster2.osd.0.smithi144.stderr:2021-03-09T12:51:42.049+0000 7f0e42124700 -1 reset not still connected to 0x5573f63c9fb0 2021-03-09T12:51:42.060 INFO:tasks.ceph.cluster2.osd.0.smithi144.stderr:2021-03-09T12:51:42.049+0000 7f0e42124700 -1 reset not still connected to 0x5573ffe04340 2021-03-09T12:51:42.060 INFO:tasks.ceph.cluster2.osd.0.smithi144.stderr:2021-03-09T12:51:42.049+0000 7f0e42124700 -1 reset not still connected to 0x557406d92820 2021-03-09T12:52:37.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8415410539564536) > weight (0.25) 2021-03-09T12:52:37.726 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T12:52:37.729 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T12:52:37.731 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T12:52:37.734 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T12:52:37.728+0000 7ffb519f8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 83092) UID: 1000 2021-03-09T12:52:37.931 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:52:40.387 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7812063499517412) > weight (0.25) 2021-03-09T12:52:40.387 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9364751328940073) > weight (0.25) 2021-03-09T12:52:40.388 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T12:52:40.388 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T12:52:40.388 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T12:52:40.396 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T12:52:40.388+0000 7fc9b7784700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 83001) UID: 1000 2021-03-09T12:52:40.590 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:52:42.733 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T12:52:42.736 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:52:42.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:52:42.737 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T12:52:42.737 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T12:52:42.737 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T12:52:42.740 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T12:52:42.740 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T12:52:47.045 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:52:47.041+0000 7f901a51c700 -1 reset not still connected to 0x5588fb01c410 2021-03-09T12:52:47.045 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:52:47.041+0000 7f901a51c700 -1 reset not still connected to 0x558904592f20 2021-03-09T12:52:47.396 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T12:52:47.399 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:52:47.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:52:47.400 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T12:52:47.400 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T12:52:47.401 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T12:52:47.404 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T12:52:47.404 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T12:53:11.135 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T12:53:11.128+0000 7f901a51c700 -1 reset not still connected to 0x558904592f20 2021-03-09T12:54:00.405 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5976618999431598) > weight (0.25) 2021-03-09T12:54:00.427 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7271650136592935) > weight (0.25) 2021-03-09T12:54:00.427 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8154460982277753) > weight (0.25) 2021-03-09T12:54:00.427 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8235127713415362) > weight (0.25) 2021-03-09T12:54:00.428 INFO:tasks.rbd_mirror_thrash:waiting for 95 secs before thrashing 2021-03-09T12:54:23.742 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T12:54:23.743 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T12:54:23.743 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T12:54:23.744 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T12:54:23.742+0000 7f497c152700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 83473) UID: 1000 2021-03-09T12:54:23.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:54:32.745 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T12:54:32.747 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:54:32.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:54:32.748 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T12:54:32.749 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T12:54:32.749 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T12:54:32.753 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T12:54:32.754 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T12:55:34.755 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9718820462337376) > weight (0.25) 2021-03-09T12:55:34.783 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8467498410269751) > weight (0.25) 2021-03-09T12:55:34.784 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3776341444805237) > weight (0.25) 2021-03-09T12:55:34.784 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T12:55:34.785 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T12:55:34.785 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T12:55:34.786 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T12:55:34.781+0000 7f57e9bb3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 83575) UID: 1000 2021-03-09T12:55:34.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:55:35.428 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.39274815261901475) > weight (0.25) 2021-03-09T12:55:35.429 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5360058259346051) > weight (0.25) 2021-03-09T12:55:35.429 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8640616072899581) > weight (0.25) 2021-03-09T12:55:35.429 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5459012351905527) > weight (0.25) 2021-03-09T12:55:35.429 INFO:tasks.rbd_mirror_thrash:waiting for 87 secs before thrashing 2021-03-09T12:55:37.786 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T12:55:37.787 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:55:37.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:55:37.788 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T12:55:37.789 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T12:55:37.789 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T12:55:37.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T12:55:37.801 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T12:56:38.803 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9362457241887628) > weight (0.25) 2021-03-09T12:56:38.834 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8074931618627702) > weight (0.25) 2021-03-09T12:56:38.834 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9452150574929856) > weight (0.25) 2021-03-09T12:56:38.835 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8918713244264744) > weight (0.25) 2021-03-09T12:56:38.835 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T12:57:02.431 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T12:57:02.434 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T12:57:02.434 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T12:57:02.435 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T12:57:02.432+0000 7fe59ff82700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 83191) UID: 1000 2021-03-09T12:57:02.635 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:57:10.435 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T12:57:10.436 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:57:10.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:57:10.437 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T12:57:10.437 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T12:57:10.437 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T12:57:10.440 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T12:57:10.441 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T12:58:15.442 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5765014963180926) > weight (0.25) 2021-03-09T12:58:15.471 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9502735322420494) > weight (0.25) 2021-03-09T12:58:15.472 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.34403204553211575) > weight (0.25) 2021-03-09T12:58:15.472 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.38007146298711203) > weight (0.25) 2021-03-09T12:58:15.472 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T12:58:34.836 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9450625097519277) > weight (0.25) 2021-03-09T12:58:34.837 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8153313233946952) > weight (0.25) 2021-03-09T12:58:34.837 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T12:58:34.838 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T12:58:34.838 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T12:58:34.839 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T12:58:34.834+0000 7f3e261e9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 82898) UID: 1000 2021-03-09T12:58:35.240 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T12:58:36.839 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T12:58:36.840 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T12:58:36.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T12:58:36.841 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T12:58:36.841 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T12:58:36.842 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T12:58:36.848 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T12:58:36.848 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T13:00:01.849 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4024084484567292) > weight (0.25) 2021-03-09T13:00:01.881 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7678757870771914) > weight (0.25) 2021-03-09T13:00:01.881 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5723423173493563) > weight (0.25) 2021-03-09T13:00:01.882 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8492563995093555) > weight (0.25) 2021-03-09T13:00:01.882 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T13:00:14.473 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4283265173617472) > weight (0.25) 2021-03-09T13:00:14.477 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.339206810561804) > weight (0.25) 2021-03-09T13:00:14.478 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8778259442801949) > weight (0.25) 2021-03-09T13:00:14.478 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.5126820723444585) > weight (0.25) 2021-03-09T13:00:14.478 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T13:01:11.884 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:01:11.891 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:01:11.891 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T13:01:11.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:01:11.887+0000 7f9e1b83c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 83866) UID: 1000 2021-03-09T13:01:12.068 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:01:12.067+0000 7f901a51c700 -1 reset not still connected to 0x5589099a0000 2021-03-09T13:01:12.069 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:01:12.067+0000 7f901a51c700 -1 reset not still connected to 0x558909d312b0 2021-03-09T13:01:12.293 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:01:19.892 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:01:19.893 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:01:19.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:01:19.894 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:01:19.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:01:19.896 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:01:19.899 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:01:19.899 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T13:02:10.479 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5759160285392688) > weight (0.25) 2021-03-09T13:02:10.481 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.37537957827280477) > weight (0.25) 2021-03-09T13:02:10.482 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.7102343919697038) > weight (0.25) 2021-03-09T13:02:10.482 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T13:02:10.483 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T13:02:10.484 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T13:02:10.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T13:02:10.478+0000 7fe5344ea700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 83374) UID: 1000 2021-03-09T13:02:10.684 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:02:16.485 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T13:02:16.487 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:02:16.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:02:16.488 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T13:02:16.488 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T13:02:16.489 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T13:02:16.492 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T13:02:16.493 INFO:tasks.rbd_mirror_thrash:waiting for 65 secs before thrashing 2021-03-09T13:02:21.229 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:02:21.225+0000 7f901a51c700 -1 reset not still connected to 0x558908c89860 2021-03-09T13:02:49.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.32552087425636356) > weight (0.25) 2021-03-09T13:02:49.905 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5880190393985821) > weight (0.25) 2021-03-09T13:02:49.905 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.49347932216695134) > weight (0.25) 2021-03-09T13:02:49.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5256896734925591) > weight (0.25) 2021-03-09T13:02:49.906 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T13:03:21.494 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2742079365865424) > weight (0.25) 2021-03-09T13:03:21.496 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8377296048141049) > weight (0.25) 2021-03-09T13:03:21.497 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.26414764499769516) > weight (0.25) 2021-03-09T13:03:21.497 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6872731009318953) > weight (0.25) 2021-03-09T13:03:21.497 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T13:03:55.907 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:03:55.944 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:03:55.944 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T13:03:55.945 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:03:55.937+0000 7f51631d7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 84262) UID: 1000 2021-03-09T13:03:56.145 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:03:56.945 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:03:56.946 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:03:56.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:03:56.947 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:03:56.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:03:56.948 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:03:56.950 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:03:56.951 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T13:04:51.498 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T13:04:51.501 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T13:04:51.501 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T13:04:51.502 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T13:04:51.500+0000 7fbfdc7e6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 84068) UID: 1000 2021-03-09T13:04:51.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:04:55.502 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T13:04:55.507 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:04:55.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:04:55.508 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T13:04:55.508 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T13:04:55.509 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T13:04:55.511 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T13:04:55.512 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T13:05:53.952 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7023789516550902) > weight (0.25) 2021-03-09T13:05:53.954 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8396878659758014) > weight (0.25) 2021-03-09T13:05:53.954 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3792597174668244) > weight (0.25) 2021-03-09T13:05:53.954 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T13:05:53.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T13:05:53.956 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T13:05:53.957 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T13:05:53.951+0000 7f2341f4f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 83969) UID: 1000 2021-03-09T13:05:54.157 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:05:58.957 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T13:05:58.958 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:05:58.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:05:58.959 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T13:05:58.959 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T13:05:58.960 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T13:05:58.964 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T13:05:58.965 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T13:06:12.513 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6804255328427766) > weight (0.25) 2021-03-09T13:06:12.516 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9747552773802352) > weight (0.25) 2021-03-09T13:06:12.516 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.25043464808812954) > weight (0.25) 2021-03-09T13:06:12.516 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.48742707124818874) > weight (0.25) 2021-03-09T13:06:12.516 INFO:tasks.rbd_mirror_thrash:waiting for 79 secs before thrashing 2021-03-09T13:07:29.966 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.48820691275239214) > weight (0.25) 2021-03-09T13:07:30.172 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9726131446912346) > weight (0.25) 2021-03-09T13:07:30.173 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T13:07:30.173 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T13:07:30.174 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T13:07:30.176 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T13:07:30.173+0000 7f686bf0f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 84161) UID: 1000 2021-03-09T13:07:30.377 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:07:31.517 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T13:07:31.518 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T13:07:31.519 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T13:07:31.519 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T13:07:31.517+0000 7ff393097700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 84554) UID: 1000 2021-03-09T13:07:31.720 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:07:32.520 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T13:07:32.520 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:07:32.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:07:32.522 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T13:07:32.522 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T13:07:32.523 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T13:07:32.527 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T13:07:32.527 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T13:07:37.175 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T13:07:37.176 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:07:37.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:07:37.177 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T13:07:37.177 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T13:07:37.177 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T13:07:37.180 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T13:07:37.181 INFO:tasks.rbd_mirror_thrash:waiting for 72 secs before thrashing 2021-03-09T13:08:02.530 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T13:08:02.527+0000 7fdda4a8d700 -1 reset not still connected to 0x55aef86f2dd0 2021-03-09T13:08:02.531 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T13:08:02.527+0000 7fdda4a8d700 -1 reset not still connected to 0x55aef86f43c0 2021-03-09T13:08:02.532 INFO:tasks.ceph.cluster1.osd.1.smithi092.stderr:2021-03-09T13:08:02.527+0000 7fdda4a8d700 -1 reset not still connected to 0x55aefb276f20 2021-03-09T13:08:49.182 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3264190194669462) > weight (0.25) 2021-03-09T13:08:49.208 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6593667760415735) > weight (0.25) 2021-03-09T13:08:49.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7512957442822378) > weight (0.25) 2021-03-09T13:08:49.209 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5217823453115491) > weight (0.25) 2021-03-09T13:08:49.209 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T13:09:09.528 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3438114921463783) > weight (0.25) 2021-03-09T13:09:09.531 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T13:09:09.532 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T13:09:09.532 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T13:09:09.533 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T13:09:09.531+0000 7f21ae424700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 82614) UID: 1000 2021-03-09T13:09:09.733 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:09:12.533 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T13:09:12.534 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:09:12.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:09:12.535 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T13:09:12.535 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T13:09:12.536 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T13:09:12.538 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T13:09:12.539 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T13:09:51.211 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7265207244631519) > weight (0.25) 2021-03-09T13:09:51.214 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5088685478899142) > weight (0.25) 2021-03-09T13:09:51.214 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T13:09:51.215 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T13:09:51.215 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T13:09:51.216 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T13:09:51.215+0000 7f01e48d9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 84839) UID: 1000 2021-03-09T13:09:51.417 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:09:53.216 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T13:09:53.216 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:09:53.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:09:53.217 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T13:09:53.217 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T13:09:53.218 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T13:09:53.230 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T13:09:53.230 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T13:10:59.232 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.43534603536281347) > weight (0.25) 2021-03-09T13:10:59.258 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.2934437831075928) > weight (0.25) 2021-03-09T13:10:59.259 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T13:10:59.259 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T13:10:59.260 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T13:10:59.261 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T13:10:59.257+0000 7ff33205f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 85032) UID: 1000 2021-03-09T13:10:59.461 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:11:00.261 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T13:11:00.262 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:11:00.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:11:00.263 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T13:11:00.263 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T13:11:00.263 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T13:11:00.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T13:11:00.267 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T13:11:02.540 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.40618343077392227) > weight (0.25) 2021-03-09T13:11:02.541 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T13:11:02.541 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T13:11:02.541 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T13:11:02.542 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T13:11:02.537+0000 7fc24c5f4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 84940) UID: 1000 2021-03-09T13:11:02.743 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:11:06.543 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T13:11:06.711 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:11:06.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:11:06.712 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T13:11:06.712 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T13:11:06.712 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T13:11:06.716 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T13:11:06.716 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T13:12:06.267 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:12:06.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:12:06.293 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T13:12:06.294 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:12:06.292+0000 7f1afe9e7700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 84455) UID: 1000 2021-03-09T13:12:06.494 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:12:15.294 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:12:15.296 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:12:15.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:12:15.297 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:12:15.298 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:12:15.298 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:12:15.301 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:12:15.301 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T13:12:34.717 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8724746509320486) > weight (0.25) 2021-03-09T13:12:34.725 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T13:12:34.726 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T13:12:34.726 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T13:12:34.727 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T13:12:34.724+0000 7f947cb56700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 85231) UID: 1000 2021-03-09T13:12:34.927 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:12:41.730 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T13:12:41.738 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:12:41.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:12:41.739 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T13:12:41.739 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T13:12:41.740 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T13:12:41.745 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T13:12:41.746 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T13:14:03.303 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2834812844881579) > weight (0.25) 2021-03-09T13:14:03.333 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T13:14:03.334 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T13:14:03.334 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T13:14:03.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T13:14:03.330+0000 7f46c2c3b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 83675) UID: 1000 2021-03-09T13:14:03.736 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:14:04.766 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:14:04.760+0000 7f901a51c700 -1 reset not still connected to 0x558905ac9c70 2021-03-09T13:14:07.336 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T13:14:07.336 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:14:07.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:14:07.337 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T13:14:07.338 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T13:14:07.339 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T13:14:07.343 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T13:14:07.345 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T13:14:26.577 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:14:26.572+0000 7f901a51c700 -1 reset not still connected to 0x55890173d8e0 2021-03-09T13:14:26.582 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:14:26.576+0000 7f901a51c700 -1 reset not still connected to 0x558901a67e10 2021-03-09T13:14:26.582 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:14:26.576+0000 7f901a51c700 -1 reset not still connected to 0x55890947b810 2021-03-09T13:14:28.747 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.3578134479268411) > weight (0.25) 2021-03-09T13:14:28.749 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2903493962267286) > weight (0.25) 2021-03-09T13:14:28.749 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8428078508697401) > weight (0.25) 2021-03-09T13:14:28.750 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.25598833358433704) > weight (0.25) 2021-03-09T13:14:28.750 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T13:14:41.978 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:14:41.971+0000 7f901a51c700 -1 reset not still connected to 0x558903e2d5a0 2021-03-09T13:14:47.646 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:14:47.643+0000 7f901a51c700 -1 reset not still connected to 0x5588f254f2b0 2021-03-09T13:15:16.346 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5204600625381105) > weight (0.25) 2021-03-09T13:15:16.351 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.706249556108442) > weight (0.25) 2021-03-09T13:15:16.352 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9605248773760647) > weight (0.25) 2021-03-09T13:15:16.352 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T13:15:16.353 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T13:15:16.354 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T13:15:16.355 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T13:15:16.349+0000 7f396a8ab700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 84649) UID: 1000 2021-03-09T13:15:16.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:15:18.354 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T13:15:18.355 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:15:18.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:15:18.356 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T13:15:18.357 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T13:15:18.358 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T13:15:18.362 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T13:15:18.362 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T13:15:29.751 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.46140833962653893) > weight (0.25) 2021-03-09T13:15:29.751 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.25904925785178823) > weight (0.25) 2021-03-09T13:15:29.752 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6196797736518527) > weight (0.25) 2021-03-09T13:15:29.752 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.6792645538261302) > weight (0.25) 2021-03-09T13:15:29.752 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-03-09T13:16:20.364 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.33295274065584823) > weight (0.25) 2021-03-09T13:16:20.595 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T13:16:20.596 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T13:16:20.597 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T13:16:20.597 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T13:16:20.597 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:16:20.599 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T13:16:20.596+0000 7fab28058700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 85515) UID: 1000 2021-03-09T13:16:20.800 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:16:26.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:16:26.701 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T13:16:26.701 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T13:16:26.702 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T13:16:26.712 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T13:16:26.713 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T13:16:37.754 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7694049382130985) > weight (0.25) 2021-03-09T13:16:37.754 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6783008683812783) > weight (0.25) 2021-03-09T13:16:37.755 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8866858680545833) > weight (0.25) 2021-03-09T13:16:37.755 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.28470811828569587) > weight (0.25) 2021-03-09T13:16:37.756 INFO:tasks.rbd_mirror_thrash:waiting for 104 secs before thrashing 2021-03-09T13:18:21.714 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7455655024040622) > weight (0.25) 2021-03-09T13:18:21.733 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9172322971962111) > weight (0.25) 2021-03-09T13:18:21.733 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.878653258048737) > weight (0.25) 2021-03-09T13:18:21.733 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7741029414233815) > weight (0.25) 2021-03-09T13:18:21.734 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T13:18:21.757 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7638552019836745) > weight (0.25) 2021-03-09T13:18:21.758 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9017274069175436) > weight (0.25) 2021-03-09T13:18:21.758 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.3442729319301312) > weight (0.25) 2021-03-09T13:18:21.758 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T13:18:21.759 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T13:18:21.760 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T13:18:21.760 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T13:18:21.754+0000 7ffa0a2b3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 84361) UID: 1000 2021-03-09T13:18:21.961 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:18:28.760 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T13:18:28.769 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:18:28.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:18:28.770 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T13:18:28.770 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T13:18:28.772 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T13:18:28.774 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T13:18:28.775 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T13:19:11.979 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:19:11.972+0000 7f901a51c700 -1 reset not still connected to 0x558903e2d5a0 2021-03-09T13:20:10.735 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9499688694722407) > weight (0.25) 2021-03-09T13:20:10.770 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T13:20:10.771 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T13:20:10.771 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T13:20:10.771 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T13:20:10.772 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:20:10.772 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T13:20:10.768+0000 7f5769803700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 85717) UID: 1000 2021-03-09T13:20:10.972 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:20:16.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:20:17.129 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T13:20:17.130 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T13:20:17.130 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T13:20:17.139 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T13:20:17.141 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T13:20:21.776 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T13:20:21.777 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T13:20:21.777 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T13:20:21.779 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T13:20:21.776+0000 7fd922b26700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 84748) UID: 1000 2021-03-09T13:20:21.978 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:20:25.778 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T13:20:25.779 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:20:25.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:20:25.779 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T13:20:25.780 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T13:20:25.780 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T13:20:25.783 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T13:20:25.783 INFO:tasks.rbd_mirror_thrash:waiting for 74 secs before thrashing 2021-03-09T13:20:41.868 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:20:41.861+0000 7f901a51c700 -1 reset not still connected to 0x55890947b810 2021-03-09T13:21:39.784 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.57112697573442) > weight (0.25) 2021-03-09T13:21:39.821 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6761036570483114) > weight (0.25) 2021-03-09T13:21:39.822 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T13:21:39.823 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T13:21:39.823 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T13:21:39.824 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T13:21:39.822+0000 7f27575fc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 83766) UID: 1000 2021-03-09T13:21:40.024 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:21:44.824 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T13:21:44.826 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:21:44.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:21:44.827 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T13:21:44.827 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T13:21:44.827 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T13:21:44.833 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T13:21:44.834 INFO:tasks.rbd_mirror_thrash:waiting for 68 secs before thrashing 2021-03-09T13:21:49.151 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:21:49.154 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:21:49.155 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T13:21:49.155 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:21:49.150+0000 7ff2c7ed1700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 85323) UID: 1000 2021-03-09T13:21:49.356 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:21:54.155 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:21:54.157 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:21:54.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:21:54.158 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:21:54.158 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:21:54.158 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:21:54.161 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:21:54.162 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T13:22:52.835 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7391022119642042) > weight (0.25) 2021-03-09T13:22:52.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7651009954775436) > weight (0.25) 2021-03-09T13:22:52.838 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.43566274316071185) > weight (0.25) 2021-03-09T13:22:52.839 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7060921246780023) > weight (0.25) 2021-03-09T13:22:52.839 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T13:22:56.017 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T13:22:56.009+0000 7f082a074700 -1 reset not still connected to 0x55a99a3ba0d0 2021-03-09T13:22:56.017 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T13:22:56.009+0000 7f082a074700 -1 reset not still connected to 0x55a99a3bccb0 2021-03-09T13:22:56.018 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T13:22:56.009+0000 7f082a074700 -1 reset not still connected to 0x55a99af5f040 2021-03-09T13:23:35.163 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6767647283168607) > weight (0.25) 2021-03-09T13:23:35.164 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3395141855743926) > weight (0.25) 2021-03-09T13:23:35.165 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.326097147354936) > weight (0.25) 2021-03-09T13:23:35.165 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7582452663857938) > weight (0.25) 2021-03-09T13:23:35.165 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T13:24:40.840 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2933121562858403) > weight (0.25) 2021-03-09T13:24:40.845 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T13:24:40.846 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T13:24:40.846 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T13:24:40.847 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T13:24:40.843+0000 7f87c0577700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 85422) UID: 1000 2021-03-09T13:24:41.048 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:24:48.847 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T13:24:48.849 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:24:48.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:24:48.849 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T13:24:48.850 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T13:24:48.851 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T13:24:48.854 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T13:24:48.854 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T13:24:58.166 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:24:58.168 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:24:58.169 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T13:24:58.170 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:24:58.167+0000 7f8a689a3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86195) UID: 1000 2021-03-09T13:24:58.370 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:25:02.170 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:25:02.171 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:25:02.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:25:02.172 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:25:02.172 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:25:02.173 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:25:02.175 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:25:02.176 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T13:26:27.178 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.41013390137461303) > weight (0.25) 2021-03-09T13:26:27.222 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5893021944178346) > weight (0.25) 2021-03-09T13:26:27.223 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T13:26:27.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T13:26:27.225 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T13:26:27.228 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T13:26:27.226+0000 7f080b3b0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 85132) UID: 1000 2021-03-09T13:26:27.629 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:26:28.227 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T13:26:28.228 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:26:28.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:26:28.228 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T13:26:28.229 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T13:26:28.229 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T13:26:28.232 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T13:26:28.232 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T13:26:30.855 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9112052075126172) > weight (0.25) 2021-03-09T13:26:30.861 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T13:26:30.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T13:26:30.862 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T13:26:30.863 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T13:26:30.857+0000 7f928ca86700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 86295) UID: 1000 2021-03-09T13:26:31.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:26:32.863 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T13:26:32.863 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:26:32.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:26:32.864 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T13:26:32.864 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T13:26:32.864 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T13:26:32.867 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T13:26:32.867 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T13:27:34.233 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:27:34.236 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:27:34.237 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T13:27:34.238 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:27:34.232+0000 7f4b54253700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86390) UID: 1000 2021-03-09T13:27:34.437 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:27:40.238 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:27:40.252 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:27:40.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:27:40.253 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:27:40.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:27:40.254 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:27:40.257 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:27:40.257 INFO:tasks.rbd_mirror_thrash:waiting for 105 secs before thrashing 2021-03-09T13:28:25.869 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8860534852019658) > weight (0.25) 2021-03-09T13:28:25.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3466392029741191) > weight (0.25) 2021-03-09T13:28:25.906 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5504955799827931) > weight (0.25) 2021-03-09T13:28:25.907 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9638357073550513) > weight (0.25) 2021-03-09T13:28:25.907 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T13:29:25.258 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.30691178354405435) > weight (0.25) 2021-03-09T13:29:25.261 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5355028714003763) > weight (0.25) 2021-03-09T13:29:25.261 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3741824338199009) > weight (0.25) 2021-03-09T13:29:25.262 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7212680024182228) > weight (0.25) 2021-03-09T13:29:25.262 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T13:29:55.908 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9697206954070662) > weight (0.25) 2021-03-09T13:29:55.942 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6834397452389159) > weight (0.25) 2021-03-09T13:29:55.942 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T13:29:55.943 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T13:29:55.943 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T13:29:55.944 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T13:29:55.942+0000 7f7f6b3b5700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 86104) UID: 1000 2021-03-09T13:29:56.145 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:30:01.944 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T13:30:01.950 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:30:01.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:30:01.951 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T13:30:01.951 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T13:30:01.952 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T13:30:01.955 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T13:30:01.955 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-03-09T13:30:48.263 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.981211376253573) > weight (0.25) 2021-03-09T13:30:48.264 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4845028873785818) > weight (0.25) 2021-03-09T13:30:48.265 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.41600882329340905) > weight (0.25) 2021-03-09T13:30:48.265 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T13:30:48.266 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T13:30:48.267 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T13:30:48.268 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T13:30:48.261+0000 7febe8cf2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 85618) UID: 1000 2021-03-09T13:30:48.668 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:30:49.479 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T13:30:49.471+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad6b3fba0 2021-03-09T13:30:49.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:30:49.473+0000 7fd527f10700 -1 rbd::mirror::ImageReplayer: 0x5617b36a3b80 [3/bd4ef0c3-a3a7-42f2-8034-c828cab83717] start: already running 2021-03-09T13:30:49.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:30:49.473+0000 7fd527f10700 -1 rbd::mirror::ImageReplayer: 0x5617b7820a00 [3/c6587b12-8826-47cf-8608-1a4b4fd5600a] start: already running 2021-03-09T13:30:49.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:30:49.473+0000 7fd527f10700 -1 rbd::mirror::ImageReplayer: 0x5617b7820500 [3/f5ac9111-3dda-4ae3-aa2c-fb3fde1f88c5] start: already running 2021-03-09T13:30:52.268 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T13:30:52.269 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:30:52.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:30:52.270 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T13:30:52.270 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T13:30:52.271 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T13:30:52.273 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T13:30:52.274 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T13:30:54.476 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:30:54.471+0000 7f901a51c700 -1 reset not still connected to 0x5588ff2d68a0 2021-03-09T13:30:54.580 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:30:54.471+0000 7f901a51c700 -1 reset not still connected to 0x558900311ad0 2021-03-09T13:30:54.580 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:30:54.471+0000 7f901a51c700 -1 reset not still connected to 0x558900311fb0 2021-03-09T13:30:54.580 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:30:54.471+0000 7f901a51c700 -1 reset not still connected to 0x558902096000 2021-03-09T13:31:19.957 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.29382111983714265) > weight (0.25) 2021-03-09T13:31:19.958 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5916693623363024) > weight (0.25) 2021-03-09T13:31:19.959 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4098813429764965) > weight (0.25) 2021-03-09T13:31:19.959 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T13:31:19.959 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T13:31:19.960 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T13:31:19.961 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T13:31:19.957+0000 7f6a44ad3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 85819) UID: 1000 2021-03-09T13:31:20.161 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:31:28.960 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T13:31:28.962 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:31:28.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:31:28.963 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T13:31:28.964 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T13:31:28.965 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T13:31:28.970 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T13:31:28.971 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T13:32:01.275 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8408271111977772) > weight (0.25) 2021-03-09T13:32:01.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.3994038824297492) > weight (0.25) 2021-03-09T13:32:01.278 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.6062089694588737) > weight (0.25) 2021-03-09T13:32:01.279 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5776559530944924) > weight (0.25) 2021-03-09T13:32:01.279 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T13:33:08.280 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4706434467258277) > weight (0.25) 2021-03-09T13:33:08.305 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.428913444183171) > weight (0.25) 2021-03-09T13:33:08.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9070968738171938) > weight (0.25) 2021-03-09T13:33:08.306 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4719632014576989) > weight (0.25) 2021-03-09T13:33:08.306 INFO:tasks.rbd_mirror_thrash:waiting for 88 secs before thrashing 2021-03-09T13:33:22.972 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2950115173521418) > weight (0.25) 2021-03-09T13:33:22.974 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T13:33:22.975 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T13:33:22.975 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T13:33:22.976 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T13:33:22.974+0000 7f966cc42700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 86581) UID: 1000 2021-03-09T13:33:23.177 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:33:28.976 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T13:33:29.599 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:33:29.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:33:29.600 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T13:33:29.601 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T13:33:29.601 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T13:33:29.609 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T13:33:29.609 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T13:34:36.307 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:34:36.310 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:34:36.311 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T13:34:36.312 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:34:36.309+0000 7fd528f12700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 86681) UID: 1000 2021-03-09T13:34:36.511 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:34:38.312 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:34:38.314 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:34:38.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:34:38.315 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:34:38.315 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:34:38.317 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:34:38.320 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:34:38.321 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T13:34:59.611 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7192774049699955) > weight (0.25) 2021-03-09T13:34:59.613 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8377065970100853) > weight (0.25) 2021-03-09T13:34:59.614 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8315870113986541) > weight (0.25) 2021-03-09T13:34:59.614 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T13:34:59.615 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T13:34:59.616 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T13:34:59.616 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T13:34:59.609+0000 7fd3f8687700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 86972) UID: 1000 2021-03-09T13:34:59.816 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:35:01.617 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T13:35:01.617 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:35:01.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:35:01.619 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T13:35:01.619 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T13:35:01.620 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T13:35:01.623 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T13:35:01.623 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T13:36:30.322 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2828747496015003) > weight (0.25) 2021-03-09T13:36:30.325 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9410952700788658) > weight (0.25) 2021-03-09T13:36:30.325 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.720534458202677) > weight (0.25) 2021-03-09T13:36:30.325 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.4950441579705356) > weight (0.25) 2021-03-09T13:36:30.325 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T13:36:48.624 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42048858448384874) > weight (0.25) 2021-03-09T13:36:48.626 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.6239391914505931) > weight (0.25) 2021-03-09T13:36:48.626 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T13:36:48.627 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T13:36:48.627 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T13:36:48.628 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T13:36:48.623+0000 7f8f90129700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 86781) UID: 1000 2021-03-09T13:36:48.829 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:36:54.629 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T13:36:54.629 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:36:54.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:36:54.630 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T13:36:54.630 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T13:36:54.631 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T13:36:54.634 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T13:36:54.634 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T13:36:54.906 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T13:36:54.903+0000 7f082a074700 -1 reset not still connected to 0x55a9c4e71cf0 2021-03-09T13:38:00.326 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:38:00.328 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:38:00.328 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T13:38:00.329 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:38:00.326+0000 7efc35500700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 87156) UID: 1000 2021-03-09T13:38:00.529 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:38:08.329 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:38:08.330 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:38:08.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:38:08.331 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:38:08.331 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:38:08.332 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:38:08.335 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:38:08.335 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T13:38:41.635 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5867505381471514) > weight (0.25) 2021-03-09T13:38:41.660 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.686596412491574) > weight (0.25) 2021-03-09T13:38:41.661 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.648419888028432) > weight (0.25) 2021-03-09T13:38:41.661 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T13:38:41.662 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T13:38:41.662 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T13:38:41.665 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T13:38:41.661+0000 7f23f9088700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 87255) UID: 1000 2021-03-09T13:38:41.866 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:38:47.663 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T13:38:47.665 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:38:47.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:38:47.666 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T13:38:47.667 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T13:38:47.668 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T13:38:47.671 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T13:38:47.671 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T13:39:25.336 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.849368476070865) > weight (0.25) 2021-03-09T13:39:25.339 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9653384727703188) > weight (0.25) 2021-03-09T13:39:25.339 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T13:39:25.340 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T13:39:25.341 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T13:39:25.342 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T13:39:25.340+0000 7fcd64e7b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 86490) UID: 1000 2021-03-09T13:39:25.542 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:39:28.342 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T13:39:28.349 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:39:28.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:39:28.350 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T13:39:28.350 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T13:39:28.351 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T13:39:28.354 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T13:39:28.354 INFO:tasks.rbd_mirror_thrash:waiting for 106 secs before thrashing 2021-03-09T13:40:41.673 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4097202846178257) > weight (0.25) 2021-03-09T13:40:41.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.563573134437373) > weight (0.25) 2021-03-09T13:40:41.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9877697398686904) > weight (0.25) 2021-03-09T13:40:41.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9506763011502603) > weight (0.25) 2021-03-09T13:40:41.676 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T13:41:14.355 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9939074549200503) > weight (0.25) 2021-03-09T13:41:14.384 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T13:41:14.385 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T13:41:14.386 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T13:41:14.387 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T13:41:14.382+0000 7f5143866700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 85914) UID: 1000 2021-03-09T13:41:14.788 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:41:21.387 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T13:41:21.387 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:41:21.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:41:21.388 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T13:41:21.389 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T13:41:21.389 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T13:41:21.392 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T13:41:21.393 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T13:42:10.677 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7886400805455087) > weight (0.25) 2021-03-09T13:42:10.679 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.30914331423830554) > weight (0.25) 2021-03-09T13:42:10.679 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.25000834477320466) > weight (0.25) 2021-03-09T13:42:10.679 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9954303152197601) > weight (0.25) 2021-03-09T13:42:10.680 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T13:43:04.393 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5784138299594865) > weight (0.25) 2021-03-09T13:43:04.396 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9992979167144556) > weight (0.25) 2021-03-09T13:43:04.396 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4517812639315618) > weight (0.25) 2021-03-09T13:43:04.397 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6843337969942432) > weight (0.25) 2021-03-09T13:43:04.397 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T13:43:12.681 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T13:43:12.685 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T13:43:12.686 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T13:43:12.688 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T13:43:12.684+0000 7f8274714700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 86013) UID: 1000 2021-03-09T13:43:12.887 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:43:15.687 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T13:43:15.696 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:43:15.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:43:15.696 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T13:43:15.697 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T13:43:15.698 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T13:43:15.701 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T13:43:15.701 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T13:43:29.852 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:43:29.846+0000 7f901a51c700 -1 reset not still connected to 0x558900c68270 2021-03-09T13:43:29.852 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:43:29.846+0000 7f901a51c700 -1 reset not still connected to 0x55890761f790 2021-03-09T13:44:31.702 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T13:44:31.705 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T13:44:31.709 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T13:44:31.710 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T13:44:31.703+0000 7f79ee6d8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 87836) UID: 1000 2021-03-09T13:44:31.907 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:44:38.710 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T13:44:38.712 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:44:38.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:44:38.713 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T13:44:38.713 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T13:44:38.714 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T13:44:38.717 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T13:44:38.717 INFO:tasks.rbd_mirror_thrash:waiting for 114 secs before thrashing 2021-03-09T13:44:41.398 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:44:41.405 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:44:41.405 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T13:44:41.406 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:44:41.403+0000 7f7d8c82e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 87443) UID: 1000 2021-03-09T13:44:41.411 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T13:44:41.404+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad5104b60 2021-03-09T13:44:41.416 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T13:44:41.408+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad5106be0 2021-03-09T13:44:41.416 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T13:44:41.408+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9eb9810 2021-03-09T13:44:41.421 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T13:44:41.416+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad9eb99b0 2021-03-09T13:44:41.421 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T13:44:41.416+0000 7fa08ad3c700 -1 reset not still connected to 0x564adcabf040 2021-03-09T13:44:41.422 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T13:44:41.416+0000 7fa08ad3c700 -1 reset not still connected to 0x564ae0e0d110 2021-03-09T13:44:41.422 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T13:44:41.416+0000 7fa08ad3c700 -1 reset not still connected to 0x564ae0e0f9b0 2021-03-09T13:44:41.606 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:44:43.406 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:44:43.407 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:44:43.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:44:43.408 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:44:43.408 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:44:43.408 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:44:43.411 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:44:43.411 INFO:tasks.rbd_mirror_thrash:waiting for 76 secs before thrashing 2021-03-09T13:45:00.992 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T13:45:00.987+0000 7f082a074700 -1 reset not still connected to 0x55a9a13ad9b0 2021-03-09T13:45:59.413 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:45:59.415 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:45:59.415 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T13:45:59.416 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:45:59.410+0000 7f9592413700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 88086) UID: 1000 2021-03-09T13:45:59.617 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:46:07.416 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:46:07.419 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:46:07.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:46:07.419 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:46:07.420 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:46:07.420 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:46:07.423 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:46:07.423 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T13:46:32.718 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T13:46:32.721 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T13:46:32.721 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T13:46:32.722 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T13:46:32.717+0000 7f1070639700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 87995) UID: 1000 2021-03-09T13:46:32.922 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:46:38.722 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T13:46:38.722 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:46:38.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:46:38.723 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T13:46:38.724 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T13:46:38.724 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T13:46:38.740 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T13:46:38.740 INFO:tasks.rbd_mirror_thrash:waiting for 98 secs before thrashing 2021-03-09T13:47:44.425 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7088988498589932) > weight (0.25) 2021-03-09T13:47:44.475 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8800317559937841) > weight (0.25) 2021-03-09T13:47:44.476 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3261884588631926) > weight (0.25) 2021-03-09T13:47:44.476 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.8121084163892023) > weight (0.25) 2021-03-09T13:47:44.477 INFO:tasks.rbd_mirror_thrash:waiting for 111 secs before thrashing 2021-03-09T13:48:16.741 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T13:48:16.744 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T13:48:16.744 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T13:48:16.745 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T13:48:16.739+0000 7f9bb1a0d700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 88287) UID: 1000 2021-03-09T13:48:16.945 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:48:17.745 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T13:48:17.746 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:48:17.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:48:17.748 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T13:48:17.748 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T13:48:17.749 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T13:48:17.757 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T13:48:17.757 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T13:49:30.759 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T13:49:30.765 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T13:49:30.766 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T13:49:30.766 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T13:49:30.762+0000 7f65d5ed3700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 88380) UID: 1000 2021-03-09T13:49:30.967 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:49:33.767 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T13:49:33.767 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:49:33.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:49:33.768 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T13:49:33.769 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T13:49:33.769 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T13:49:33.772 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T13:49:33.772 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T13:49:35.487 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5203690074675013) > weight (0.25) 2021-03-09T13:49:35.504 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.6048352575967373) > weight (0.25) 2021-03-09T13:49:35.504 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T13:49:35.505 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T13:49:35.505 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T13:49:35.506 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T13:49:35.506 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:49:35.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T13:49:35.502+0000 7fcd16002700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 87633) UID: 1000 2021-03-09T13:49:35.908 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:49:41.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:49:41.611 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T13:49:41.611 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T13:49:41.613 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T13:49:41.616 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T13:49:41.617 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T13:50:42.619 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:50:42.661 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:50:42.662 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T13:50:42.663 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:50:42.661+0000 7ff48e816700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 88188) UID: 1000 2021-03-09T13:50:42.862 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:50:45.664 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:50:45.664 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:50:45.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:50:45.666 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:50:45.666 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:50:45.667 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:50:45.674 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:50:45.675 INFO:tasks.rbd_mirror_thrash:waiting for 107 secs before thrashing 2021-03-09T13:51:26.774 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4250871715257637) > weight (0.25) 2021-03-09T13:51:26.775 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7722006511000015) > weight (0.25) 2021-03-09T13:51:26.775 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T13:51:26.776 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T13:51:26.776 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T13:51:26.777 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T13:51:26.776+0000 7f20c7007700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 87351) UID: 1000 2021-03-09T13:51:26.978 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:51:28.777 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T13:51:28.778 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:51:28.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:51:28.779 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T13:51:28.779 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T13:51:28.779 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T13:51:28.782 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T13:51:28.782 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T13:51:47.898 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:51:47.891+0000 7f901a51c700 -1 reset not still connected to 0x5589020976c0 2021-03-09T13:52:32.676 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7832018887616388) > weight (0.25) 2021-03-09T13:52:32.882 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.48933071248360793) > weight (0.25) 2021-03-09T13:52:32.883 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5135512568701026) > weight (0.25) 2021-03-09T13:52:32.883 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.32195587528321645) > weight (0.25) 2021-03-09T13:52:32.884 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T13:52:41.784 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5260735180234966) > weight (0.25) 2021-03-09T13:52:41.784 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5316167949374956) > weight (0.25) 2021-03-09T13:52:41.784 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T13:52:41.785 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T13:52:41.785 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T13:52:41.786 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T13:52:41.783+0000 7f6647c64700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 88763) UID: 1000 2021-03-09T13:52:41.987 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:52:50.787 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T13:52:50.787 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:52:50.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:52:50.788 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T13:52:50.788 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T13:52:50.789 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T13:52:50.792 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T13:52:50.793 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T13:54:14.794 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T13:54:14.858 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T13:54:14.860 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T13:54:14.861 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T13:54:14.853+0000 7f975699a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 88473) UID: 1000 2021-03-09T13:54:15.059 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:54:22.861 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T13:54:22.862 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:54:22.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:54:22.863 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T13:54:22.863 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T13:54:22.865 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T13:54:22.868 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T13:54:22.868 INFO:tasks.rbd_mirror_thrash:waiting for 119 secs before thrashing 2021-03-09T13:54:27.885 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6052605647251452) > weight (0.25) 2021-03-09T13:54:27.886 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T13:54:27.886 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T13:54:27.887 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T13:54:27.888 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T13:54:27.885+0000 7ff20ec6b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 87735) UID: 1000 2021-03-09T13:54:28.088 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:54:36.888 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T13:54:36.888 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:54:36.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:54:36.889 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T13:54:36.889 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T13:54:36.889 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T13:54:36.892 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T13:54:36.892 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T13:56:21.869 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5352484198395384) > weight (0.25) 2021-03-09T13:56:21.904 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8361779860925694) > weight (0.25) 2021-03-09T13:56:21.905 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T13:56:21.906 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T13:56:21.907 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T13:56:21.908 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T13:56:21.903+0000 7f8aeb131700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 88855) UID: 1000 2021-03-09T13:56:22.108 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:56:23.908 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T13:56:23.909 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:56:23.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:56:23.910 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T13:56:23.910 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T13:56:23.911 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T13:56:23.914 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T13:56:23.914 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T13:56:29.893 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.793323343821008) > weight (0.25) 2021-03-09T13:56:29.899 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7763083654099164) > weight (0.25) 2021-03-09T13:56:29.899 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4223399004629753) > weight (0.25) 2021-03-09T13:56:29.900 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.6373196629034549) > weight (0.25) 2021-03-09T13:56:29.900 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T13:57:32.916 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T13:57:32.918 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T13:57:32.918 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T13:57:32.919 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T13:57:32.914+0000 7f724ea3c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 88947) UID: 1000 2021-03-09T13:57:33.120 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:57:33.920 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T13:57:33.921 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:57:33.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:57:33.923 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T13:57:33.923 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T13:57:33.924 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T13:57:33.930 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T13:57:33.930 INFO:tasks.rbd_mirror_thrash:waiting for 71 secs before thrashing 2021-03-09T13:57:42.901 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.36219509277751505) > weight (0.25) 2021-03-09T13:57:42.902 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.40173634856625395) > weight (0.25) 2021-03-09T13:57:42.902 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T13:57:42.903 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T13:57:42.903 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T13:57:42.904 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T13:57:42.898+0000 7f1092dbf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 88564) UID: 1000 2021-03-09T13:57:43.104 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:57:50.904 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T13:57:50.905 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:57:50.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:57:50.905 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T13:57:50.906 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T13:57:50.906 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T13:57:50.909 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T13:57:50.909 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T13:58:10.197 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T13:58:10.192+0000 7f901a51c700 -1 reset not still connected to 0x558900dc8d80 2021-03-09T13:58:44.933 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6882411137026934) > weight (0.25) 2021-03-09T13:58:44.941 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T13:58:44.941 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T13:58:44.943 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T13:58:44.944 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T13:58:44.941+0000 7f2bb3a23700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 87065) UID: 1000 2021-03-09T13:58:45.090 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T13:58:45.085+0000 7f082a074700 -1 reset not still connected to 0x55a99f7e5ad0 2021-03-09T13:58:45.143 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:58:46.943 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T13:58:46.948 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:58:46.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:58:46.949 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T13:58:46.950 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T13:58:46.950 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T13:58:46.953 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T13:58:46.953 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T13:59:03.910 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T13:59:03.912 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T13:59:03.913 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T13:59:03.914 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T13:59:03.912+0000 7fa652763700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 88664) UID: 1000 2021-03-09T13:59:04.114 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:59:09.914 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T13:59:09.921 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:59:09.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:59:09.922 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T13:59:09.922 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T13:59:09.923 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T13:59:09.926 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T13:59:09.926 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T13:59:50.954 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4870274133069331) > weight (0.25) 2021-03-09T13:59:50.961 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T13:59:50.962 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T13:59:50.963 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T13:59:50.964 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T13:59:50.959+0000 7fdf492c9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 89424) UID: 1000 2021-03-09T13:59:51.164 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T13:59:56.964 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T13:59:56.971 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T13:59:56.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T13:59:56.972 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T13:59:56.973 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T13:59:56.973 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T13:59:56.976 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T13:59:56.976 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T14:00:26.927 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T14:00:26.936 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T14:00:26.937 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T14:00:26.937 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T14:00:26.935+0000 7f24a4808700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 89515) UID: 1000 2021-03-09T14:00:27.138 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:00:35.938 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T14:00:35.945 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:00:35.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:00:35.946 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T14:00:35.947 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T14:00:35.952 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T14:00:35.955 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T14:00:35.955 INFO:tasks.rbd_mirror_thrash:waiting for 109 secs before thrashing 2021-03-09T14:00:59.977 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8177781669814241) > weight (0.25) 2021-03-09T14:00:59.981 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9394637759795711) > weight (0.25) 2021-03-09T14:00:59.981 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.31993146535602646) > weight (0.25) 2021-03-09T14:00:59.981 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T14:00:59.982 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T14:00:59.983 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T14:00:59.985 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T14:00:59.978+0000 7f1beb135700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 87542) UID: 1000 2021-03-09T14:01:00.384 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:01:08.985 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T14:01:08.986 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:01:08.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:01:08.988 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T14:01:08.988 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T14:01:08.990 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T14:01:08.993 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T14:01:08.994 INFO:tasks.rbd_mirror_thrash:waiting for 112 secs before thrashing 2021-03-09T14:02:24.956 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.49792080691294804) > weight (0.25) 2021-03-09T14:02:24.978 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4495315162401202) > weight (0.25) 2021-03-09T14:02:24.978 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.89397062503624) > weight (0.25) 2021-03-09T14:02:24.979 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7966281786538162) > weight (0.25) 2021-03-09T14:02:24.979 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T14:03:00.995 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5449111367907163) > weight (0.25) 2021-03-09T14:03:01.028 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.2758184323498444) > weight (0.25) 2021-03-09T14:03:01.029 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T14:03:01.031 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T14:03:01.033 INFO:tasks.rbd_mirror_thrash:waiting for 4 secs before reviving daemons 2021-03-09T14:03:01.035 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T14:03:01.028+0000 7f64665a4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 89141) UID: 1000 2021-03-09T14:03:01.233 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:03:03.405 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:03:03.400+0000 7f901a51c700 -1 reset not still connected to 0x558903e2bad0 2021-03-09T14:03:05.035 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T14:03:05.043 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:03:05.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:03:05.044 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T14:03:05.044 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T14:03:05.045 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T14:03:05.049 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T14:03:05.049 INFO:tasks.rbd_mirror_thrash:waiting for 103 secs before thrashing 2021-03-09T14:03:31.228 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:03:31.223+0000 7f901a51c700 -1 reset not still connected to 0x5589049282f0 2021-03-09T14:04:17.980 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7728272782344996) > weight (0.25) 2021-03-09T14:04:17.982 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T14:04:17.983 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T14:04:17.984 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T14:04:17.985 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T14:04:17.983+0000 7f2383e17700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 89038) UID: 1000 2021-03-09T14:04:18.185 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:04:25.984 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T14:04:25.985 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:04:25.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:04:25.986 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T14:04:25.986 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T14:04:25.987 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T14:04:26.001 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T14:04:26.002 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T14:04:31.541 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T14:04:31.537+0000 7fa08ad3c700 -1 reset not still connected to 0x564ad717c820 2021-03-09T14:04:31.542 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T14:04:31.537+0000 7fa08ad3c700 -1 reset not still connected to 0x564ae0e0cc30 2021-03-09T14:04:48.050 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8052762133554479) > weight (0.25) 2021-03-09T14:04:48.052 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8160443936118895) > weight (0.25) 2021-03-09T14:04:48.052 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.38563550824994963) > weight (0.25) 2021-03-09T14:04:48.052 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T14:04:48.053 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T14:04:48.053 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T14:04:48.054 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T14:04:48.050+0000 7fe19bbe2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 89806) UID: 1000 2021-03-09T14:04:48.255 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:04:51.055 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T14:04:51.055 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:04:51.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:04:51.056 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T14:04:51.056 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T14:04:51.057 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T14:04:51.059 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T14:04:51.060 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-03-09T14:05:58.003 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T14:05:58.007 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T14:05:58.008 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T14:05:58.008 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T14:05:58.005+0000 7f77a6b07700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 89706) UID: 1000 2021-03-09T14:05:58.209 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:06:06.009 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T14:06:06.009 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:06:06.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:06:06.010 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T14:06:06.010 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T14:06:06.011 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T14:06:06.014 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T14:06:06.014 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T14:06:06.061 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8600495785190724) > weight (0.25) 2021-03-09T14:06:06.061 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T14:06:06.062 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T14:06:06.062 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T14:06:06.063 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T14:06:06.057+0000 7f5d06cbe700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 89614) UID: 1000 2021-03-09T14:06:06.263 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:06:10.543 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:06:10.538+0000 7f901a51c700 -1 reset not still connected to 0x5588f8d66560 2021-03-09T14:06:10.543 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:06:10.538+0000 7f901a51c700 -1 reset not still connected to 0x5588f928aa40 2021-03-09T14:06:10.548 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:06:10.538+0000 7f901a51c700 -1 reset not still connected to 0x558900dc7ad0 2021-03-09T14:06:10.549 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:06:10.538+0000 7f901a51c700 -1 reset not still connected to 0x558900dc8d80 2021-03-09T14:06:10.549 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:06:10.538+0000 7f901a51c700 -1 reset not still connected to 0x558902096c30 2021-03-09T14:06:13.063 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T14:06:13.063 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:06:13.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:06:13.064 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T14:06:13.064 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T14:06:13.065 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T14:06:13.078 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T14:06:13.078 INFO:tasks.rbd_mirror_thrash:waiting for 89 secs before thrashing 2021-03-09T14:07:42.079 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8906904797985576) > weight (0.25) 2021-03-09T14:07:42.082 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.7601571776061342) > weight (0.25) 2021-03-09T14:07:42.083 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.9360228202911733) > weight (0.25) 2021-03-09T14:07:42.083 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8686534254721181) > weight (0.25) 2021-03-09T14:07:42.083 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T14:07:48.015 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7150722857732129) > weight (0.25) 2021-03-09T14:07:48.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.92638589102111) > weight (0.25) 2021-03-09T14:07:48.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.37647297252221834) > weight (0.25) 2021-03-09T14:07:48.016 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7338781935038058) > weight (0.25) 2021-03-09T14:07:48.017 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T14:08:48.017 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.566609169461619) > weight (0.25) 2021-03-09T14:08:48.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.948690543201684) > weight (0.25) 2021-03-09T14:08:48.025 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.9347433315833903) > weight (0.25) 2021-03-09T14:08:48.026 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T14:08:48.027 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T14:08:48.028 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T14:08:48.029 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T14:08:48.026+0000 7f8622859700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 86873) UID: 1000 2021-03-09T14:08:48.429 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:08:49.333 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:08:49.330+0000 7f901a51c700 -1 reset not still connected to 0x5588fe49e970 2021-03-09T14:08:49.334 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:08:49.330+0000 7f901a51c700 -1 reset not still connected to 0x5588fe49ebe0 2021-03-09T14:08:49.336 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:08:49.334+0000 7f901a51c700 -1 reset not still connected to 0x558900313b50 2021-03-09T14:08:53.029 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T14:08:53.031 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:08:53.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:08:53.031 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T14:08:53.032 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T14:08:53.033 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T14:08:53.036 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T14:08:53.036 INFO:tasks.rbd_mirror_thrash:waiting for 70 secs before thrashing 2021-03-09T14:09:23.084 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7467770495121733) > weight (0.25) 2021-03-09T14:09:23.087 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4429788237223551) > weight (0.25) 2021-03-09T14:09:23.088 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5953581826827392) > weight (0.25) 2021-03-09T14:09:23.088 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.3369257199257173) > weight (0.25) 2021-03-09T14:09:23.088 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T14:10:03.037 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.5056315087540405) > weight (0.25) 2021-03-09T14:10:03.039 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9766764777812397) > weight (0.25) 2021-03-09T14:10:03.040 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.33078775535329275) > weight (0.25) 2021-03-09T14:10:03.040 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5202766279441183) > weight (0.25) 2021-03-09T14:10:03.040 INFO:tasks.rbd_mirror_thrash:waiting for 73 secs before thrashing 2021-03-09T14:10:40.090 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T14:10:40.099 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T14:10:40.100 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T14:10:40.101 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T14:10:40.096+0000 7fe73546a700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 89232) UID: 1000 2021-03-09T14:10:40.301 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:10:41.281 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T14:10:41.280+0000 7f082a074700 -1 reset not still connected to 0x55a9acd57520 2021-03-09T14:10:41.282 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T14:10:41.280+0000 7f082a074700 -1 reset not still connected to 0x55a9acd59cf0 2021-03-09T14:10:41.282 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T14:10:41.280+0000 7f082a074700 -1 reset not still connected to 0x55a9c4e70a40 2021-03-09T14:10:45.101 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T14:10:45.101 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:10:45.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:10:45.102 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T14:10:45.103 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T14:10:45.103 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T14:10:45.106 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T14:10:45.107 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T14:11:16.041 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7209007820919255) > weight (0.25) 2021-03-09T14:11:16.047 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8084515211390867) > weight (0.25) 2021-03-09T14:11:16.047 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3788897616508108) > weight (0.25) 2021-03-09T14:11:16.047 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5081188253969521) > weight (0.25) 2021-03-09T14:11:16.048 INFO:tasks.rbd_mirror_thrash:waiting for 63 secs before thrashing 2021-03-09T14:12:08.107 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.263088298530917) > weight (0.25) 2021-03-09T14:12:08.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.461408104913291) > weight (0.25) 2021-03-09T14:12:08.111 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T14:12:08.112 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T14:12:08.112 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T14:12:08.113 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T14:12:08.111+0000 7f3c38ef9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 89897) UID: 1000 2021-03-09T14:12:08.314 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:12:10.113 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T14:12:10.114 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:12:10.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:12:10.115 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T14:12:10.115 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T14:12:10.115 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T14:12:10.123 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T14:12:10.123 INFO:tasks.rbd_mirror_thrash:waiting for 96 secs before thrashing 2021-03-09T14:12:19.049 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6662445172859014) > weight (0.25) 2021-03-09T14:12:19.049 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9080488697932938) > weight (0.25) 2021-03-09T14:12:19.049 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T14:12:19.050 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T14:12:19.050 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T14:12:19.051 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T14:12:19.047+0000 7fd8fc0de700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 89323) UID: 1000 2021-03-09T14:12:19.252 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:12:21.051 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T14:12:21.052 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:12:21.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:12:21.052 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T14:12:21.053 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T14:12:21.053 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T14:12:21.056 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T14:12:21.056 INFO:tasks.rbd_mirror_thrash:waiting for 102 secs before thrashing 2021-03-09T14:12:53.621 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:12:53.539+0000 7f901a51c700 -1 reset not still connected to 0x5588f24dcf70 2021-03-09T14:12:53.622 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:12:53.539+0000 7f901a51c700 -1 reset not still connected to 0x558907b941a0 2021-03-09T14:13:46.125 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7438141476610366) > weight (0.25) 2021-03-09T14:13:46.125 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T14:13:46.126 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T14:13:46.127 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T14:13:46.128 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T14:13:46.125+0000 7f5ca5146700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 90282) UID: 1000 2021-03-09T14:13:46.328 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:13:48.127 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T14:13:48.128 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:13:48.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:13:48.129 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T14:13:48.129 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T14:13:48.129 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T14:13:48.132 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T14:13:48.132 INFO:tasks.rbd_mirror_thrash:waiting for 64 secs before thrashing 2021-03-09T14:14:03.057 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7880224725059783) > weight (0.25) 2021-03-09T14:14:03.065 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T14:14:03.066 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T14:14:03.067 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T14:14:03.068 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T14:14:03.065+0000 7ff79ccec700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 89990) UID: 1000 2021-03-09T14:14:03.468 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:14:11.068 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T14:14:11.073 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:14:11.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:14:11.074 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T14:14:11.074 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T14:14:11.074 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T14:14:11.077 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T14:14:11.077 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T14:14:19.388 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:14:19.384+0000 7f901a51c700 -1 reset not still connected to 0x558907b95ad0 2021-03-09T14:14:40.362 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:14:40.356+0000 7f901a51c700 -1 reset not still connected to 0x558907b94270 2021-03-09T14:14:40.362 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:14:40.356+0000 7f901a51c700 -1 reset not still connected to 0x558907b95a00 2021-03-09T14:14:52.133 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T14:14:52.141 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T14:14:52.142 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T14:14:52.143 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T14:14:52.140+0000 7f8c8fbee700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 90477) UID: 1000 2021-03-09T14:14:52.343 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:14:54.143 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T14:14:54.144 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:14:54.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:14:54.145 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T14:14:54.145 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T14:14:54.145 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T14:14:54.148 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T14:14:54.148 INFO:tasks.rbd_mirror_thrash:waiting for 83 secs before thrashing 2021-03-09T14:15:59.078 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7001300491567842) > weight (0.25) 2021-03-09T14:15:59.090 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T14:15:59.091 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T14:15:59.092 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T14:15:59.093 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T14:15:59.087+0000 7f8d532f4700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 90852) UID: 1000 2021-03-09T14:15:59.292 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:16:06.093 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T14:16:06.094 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:16:06.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:16:06.095 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T14:16:06.095 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T14:16:06.095 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T14:16:06.098 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T14:16:06.099 INFO:tasks.rbd_mirror_thrash:waiting for 61 secs before thrashing 2021-03-09T14:16:17.150 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.384143069427322) > weight (0.25) 2021-03-09T14:16:17.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8889401210228524) > weight (0.25) 2021-03-09T14:16:17.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6902429726403377) > weight (0.25) 2021-03-09T14:16:17.160 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.9464516386389877) > weight (0.25) 2021-03-09T14:16:17.160 INFO:tasks.rbd_mirror_thrash:waiting for 93 secs before thrashing 2021-03-09T14:16:31.506 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:16:31.501+0000 7f901a51c700 -1 reset not still connected to 0x5589022bb040 2021-03-09T14:16:31.508 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:16:31.505+0000 7f901a51c700 -1 reset not still connected to 0x558907b974d0 2021-03-09T14:16:31.508 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:16:31.505+0000 7f901a51c700 -1 reset not still connected to 0x5589091a95f0 2021-03-09T14:17:07.100 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8430386367424243) > weight (0.25) 2021-03-09T14:17:07.109 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.386041180204881) > weight (0.25) 2021-03-09T14:17:07.109 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4868014994478156) > weight (0.25) 2021-03-09T14:17:07.110 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7190920472170584) > weight (0.25) 2021-03-09T14:17:07.110 INFO:tasks.rbd_mirror_thrash:waiting for 66 secs before thrashing 2021-03-09T14:17:50.162 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.9858187203462763) > weight (0.25) 2021-03-09T14:17:50.173 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T14:17:50.174 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T14:17:50.174 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T14:17:50.175 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T14:17:50.173+0000 7fab00cb8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 90760) UID: 1000 2021-03-09T14:17:50.375 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:17:51.175 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T14:17:51.176 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:17:51.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:17:51.177 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T14:17:51.178 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T14:17:51.178 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T14:17:51.181 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T14:17:51.182 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T14:18:13.111 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.7777979433280628) > weight (0.25) 2021-03-09T14:18:13.119 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.9156680325002172) > weight (0.25) 2021-03-09T14:18:13.120 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.5421421044617529) > weight (0.25) 2021-03-09T14:18:13.120 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.3 2021-03-09T14:18:13.120 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Sent signal 15 2021-03-09T14:18:13.121 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T14:18:13.122 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:2021-03-09T14:18:13.117+0000 7f785ecdb700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 (PID: 90375) UID: 1000 2021-03-09T14:18:13.322 INFO:tasks.rbd_mirror.cluster1.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:18:15.122 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.3 2021-03-09T14:18:15.122 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:18:15.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:18:15.123 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.3 2021-03-09T14:18:15.123 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Restarting daemon 2021-03-09T14:18:15.124 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.3 2021-03-09T14:18:15.127 INFO:tasks.rbd_mirror.cluster1.client.mirror.3:Started 2021-03-09T14:18:15.127 INFO:tasks.rbd_mirror_thrash:waiting for 115 secs before thrashing 2021-03-09T14:19:13.182 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.42888857938775415) > weight (0.25) 2021-03-09T14:19:13.190 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9077483661062036) > weight (0.25) 2021-03-09T14:19:13.190 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8202996928647348) > weight (0.25) 2021-03-09T14:19:13.191 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4354492804742932) > weight (0.25) 2021-03-09T14:19:13.191 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T14:19:21.295 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T14:19:21.287+0000 7f082a074700 -1 reset not still connected to 0x55a9acd57520 2021-03-09T14:19:21.296 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T14:19:21.287+0000 7f082a074700 -1 reset not still connected to 0x55a9acd59cf0 2021-03-09T14:19:21.296 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T14:19:21.287+0000 7f082a074700 -1 reset not still connected to 0x55a9c4e70a40 2021-03-09T14:20:10.128 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2800288683653238) > weight (0.25) 2021-03-09T14:20:10.140 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8350441699395565) > weight (0.25) 2021-03-09T14:20:10.141 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7506285888403063) > weight (0.25) 2021-03-09T14:20:10.141 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7910543526178095) > weight (0.25) 2021-03-09T14:20:10.141 INFO:tasks.rbd_mirror_thrash:waiting for 118 secs before thrashing 2021-03-09T14:21:06.192 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.36493499794498563) > weight (0.25) 2021-03-09T14:21:06.204 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4510893729618759) > weight (0.25) 2021-03-09T14:21:06.204 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.6232749801669277) > weight (0.25) 2021-03-09T14:21:06.204 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8209217053082177) > weight (0.25) 2021-03-09T14:21:06.204 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T14:22:08.142 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.554482199973697) > weight (0.25) 2021-03-09T14:22:08.186 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T14:22:08.188 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T14:22:08.189 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T14:22:08.190 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T14:22:08.185+0000 7f81fa03e700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 91046) UID: 1000 2021-03-09T14:22:08.390 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:22:13.206 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.44919080701408665) > weight (0.25) 2021-03-09T14:22:13.206 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T14:22:13.207 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T14:22:13.207 INFO:tasks.rbd_mirror_thrash:waiting for 6 secs before reviving daemons 2021-03-09T14:22:13.208 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T14:22:13.205+0000 7fbfd61d9700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 91148) UID: 1000 2021-03-09T14:22:13.408 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:22:14.190 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T14:22:14.190 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:22:14.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:22:14.191 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T14:22:14.191 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T14:22:14.192 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T14:22:14.195 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T14:22:14.195 INFO:tasks.rbd_mirror_thrash:waiting for 62 secs before thrashing 2021-03-09T14:22:19.208 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T14:22:19.209 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:22:19.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:22:19.210 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T14:22:19.210 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T14:22:19.211 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T14:22:19.213 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T14:22:19.214 INFO:tasks.rbd_mirror_thrash:waiting for 117 secs before thrashing 2021-03-09T14:23:16.196 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9208128582275877) > weight (0.25) 2021-03-09T14:23:16.209 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T14:23:16.210 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T14:23:16.211 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T14:23:16.212 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T14:23:16.207+0000 7f6ae6ca6700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 91344) UID: 1000 2021-03-09T14:23:16.412 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:23:21.212 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T14:23:21.220 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:23:21.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:23:21.221 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T14:23:21.221 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T14:23:21.221 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T14:23:21.224 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T14:23:21.224 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T14:24:16.215 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.2948099857943739) > weight (0.25) 2021-03-09T14:24:16.239 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.47023672749108025) > weight (0.25) 2021-03-09T14:24:16.239 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8846482896027427) > weight (0.25) 2021-03-09T14:24:16.240 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8301554698150212) > weight (0.25) 2021-03-09T14:24:16.240 INFO:tasks.rbd_mirror_thrash:waiting for 90 secs before thrashing 2021-03-09T14:24:53.225 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9879415274541193) > weight (0.25) 2021-03-09T14:24:53.236 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.4388418614030103) > weight (0.25) 2021-03-09T14:24:53.236 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T14:24:53.237 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T14:24:53.238 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T14:24:53.239 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T14:24:53.234+0000 7f33ce8d0700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 90659) UID: 1000 2021-03-09T14:24:53.438 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:25:00.242 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T14:25:00.249 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:25:00.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:25:00.250 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T14:25:00.250 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T14:25:00.250 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T14:25:00.253 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T14:25:00.253 INFO:tasks.rbd_mirror_thrash:waiting for 78 secs before thrashing 2021-03-09T14:25:06.766 INFO:tasks.ceph.cluster1.osd.0.smithi092.stderr:2021-03-09T14:25:06.750+0000 7fa08ad3c700 -1 reset not still connected to 0x564ae0e0f740 2021-03-09T14:25:46.241 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8593063769797452) > weight (0.25) 2021-03-09T14:25:46.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8575956295969308) > weight (0.25) 2021-03-09T14:25:46.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8395113397829552) > weight (0.25) 2021-03-09T14:25:46.252 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.7021835574314946) > weight (0.25) 2021-03-09T14:25:46.253 INFO:tasks.rbd_mirror_thrash:waiting for 108 secs before thrashing 2021-03-09T14:26:18.255 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9432478659683444) > weight (0.25) 2021-03-09T14:26:18.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.7369190342247748) > weight (0.25) 2021-03-09T14:26:18.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8313887994498206) > weight (0.25) 2021-03-09T14:26:18.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5543316746996548) > weight (0.25) 2021-03-09T14:26:18.282 INFO:tasks.rbd_mirror_thrash:waiting for 82 secs before thrashing 2021-03-09T14:27:34.254 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T14:27:34.263 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T14:27:34.264 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T14:27:34.265 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T14:27:34.263+0000 7f4b3b5d8700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 90951) UID: 1000 2021-03-09T14:27:34.465 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:27:37.265 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T14:27:37.266 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:27:37.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:27:37.267 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T14:27:37.267 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T14:27:37.267 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T14:27:37.270 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T14:27:37.270 INFO:tasks.rbd_mirror_thrash:waiting for 84 secs before thrashing 2021-03-09T14:27:40.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.4668133237009183) > weight (0.25) 2021-03-09T14:27:40.283 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T14:27:40.283 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T14:27:40.284 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T14:27:40.284 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T14:27:40.284 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:27:40.285 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T14:27:40.279+0000 7f9c9872b700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 91534) UID: 1000 2021-03-09T14:27:40.485 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:27:46.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:27:46.395 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T14:27:46.396 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T14:27:46.396 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T14:27:46.399 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T14:27:46.399 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T14:28:53.400 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.8704058366992085) > weight (0.25) 2021-03-09T14:28:53.410 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.994982338098582) > weight (0.25) 2021-03-09T14:28:53.411 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8385314940764732) > weight (0.25) 2021-03-09T14:28:53.411 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.7538379455758094) > weight (0.25) 2021-03-09T14:28:53.411 INFO:tasks.rbd_mirror_thrash:waiting for 67 secs before thrashing 2021-03-09T14:29:01.271 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6803666587124498) > weight (0.25) 2021-03-09T14:29:01.272 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.9767964801937843) > weight (0.25) 2021-03-09T14:29:01.272 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.5034153095317739) > weight (0.25) 2021-03-09T14:29:01.272 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.3 2021-03-09T14:29:01.273 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Sent signal 15 2021-03-09T14:29:01.274 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T14:29:01.274 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:2021-03-09T14:29:01.270+0000 7f55ea39f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 (PID: 90089) UID: 1000 2021-03-09T14:29:01.475 INFO:tasks.rbd_mirror.cluster2.client.mirror.3.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:29:03.275 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.3 2021-03-09T14:29:03.275 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:29:03.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:29:03.276 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.3 2021-03-09T14:29:03.276 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Restarting daemon 2021-03-09T14:29:03.277 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.3 2021-03-09T14:29:03.279 INFO:tasks.rbd_mirror.cluster2.client.mirror.3:Started 2021-03-09T14:29:03.280 INFO:tasks.rbd_mirror_thrash:waiting for 60 secs before thrashing 2021-03-09T14:30:00.413 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.2964398091822673) > weight (0.25) 2021-03-09T14:30:00.421 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.5206168520878379) > weight (0.25) 2021-03-09T14:30:00.421 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.3876008811603271) > weight (0.25) 2021-03-09T14:30:00.422 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.40424945165086434) > weight (0.25) 2021-03-09T14:30:00.422 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T14:30:03.281 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6884321976575172) > weight (0.25) 2021-03-09T14:30:03.282 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3390720787967936) > weight (0.25) 2021-03-09T14:30:03.282 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T14:30:03.283 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T14:30:03.284 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T14:30:03.285 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T14:30:03.281+0000 7f6f5f7dc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 90568) UID: 1000 2021-03-09T14:30:03.485 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:30:06.285 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T14:30:06.295 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:30:06.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:30:06.296 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T14:30:06.296 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T14:30:06.297 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T14:30:06.300 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T14:30:06.300 INFO:tasks.rbd_mirror_thrash:waiting for 113 secs before thrashing 2021-03-09T14:31:56.423 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9360910049594608) > weight (0.25) 2021-03-09T14:31:56.475 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.8086157211544045) > weight (0.25) 2021-03-09T14:31:56.476 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.2 2021-03-09T14:31:56.477 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Sent signal 15 2021-03-09T14:31:56.477 INFO:tasks.rbd_mirror_thrash:waiting for 7 secs before reviving daemons 2021-03-09T14:31:56.478 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T14:31:56.475+0000 7f84a513c700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 (PID: 91633) UID: 1000 2021-03-09T14:31:56.678 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:31:59.301 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.6117483844151806) > weight (0.25) 2021-03-09T14:31:59.301 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T14:31:59.302 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T14:31:59.302 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T14:31:59.303 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T14:31:59.299+0000 7f98d4b64700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 91435) UID: 1000 2021-03-09T14:31:59.703 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:32:01.303 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T14:32:01.304 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:32:01.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:32:01.305 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T14:32:01.305 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T14:32:01.305 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T14:32:01.308 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T14:32:01.308 INFO:tasks.rbd_mirror_thrash:waiting for 110 secs before thrashing 2021-03-09T14:32:03.478 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.2 2021-03-09T14:32:03.479 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:32:03.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:32:03.479 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.2 2021-03-09T14:32:03.480 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Restarting daemon 2021-03-09T14:32:03.480 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.2 2021-03-09T14:32:03.483 INFO:tasks.rbd_mirror.cluster1.client.mirror.2:Started 2021-03-09T14:32:03.483 INFO:tasks.rbd_mirror_thrash:waiting for 116 secs before thrashing 2021-03-09T14:32:27.118 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:32:27.113+0000 7f901a51c700 -1 reset not still connected to 0x55890343fa80 2021-03-09T14:32:27.120 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:32:27.117+0000 7f901a51c700 -1 reset not still connected to 0x558907b96220 2021-03-09T14:32:27.121 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:32:27.117+0000 7f901a51c700 -1 reset not still connected to 0x558907b970c0 2021-03-09T14:33:51.309 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5524703841796464) > weight (0.25) 2021-03-09T14:33:51.321 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.4407979560354305) > weight (0.25) 2021-03-09T14:33:51.322 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.2 2021-03-09T14:33:51.322 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Sent signal 15 2021-03-09T14:33:51.324 INFO:tasks.rbd_mirror_thrash:waiting for 5 secs before reviving daemons 2021-03-09T14:33:51.325 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:2021-03-09T14:33:51.321+0000 7f97393dc700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 (PID: 92021) UID: 1000 2021-03-09T14:33:51.524 INFO:tasks.rbd_mirror.cluster2.client.mirror.2.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:33:56.325 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.2 2021-03-09T14:33:56.330 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:33:56.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:33:56.331 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.2 2021-03-09T14:33:56.331 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Restarting daemon 2021-03-09T14:33:56.332 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.2 2021-03-09T14:33:56.336 INFO:tasks.rbd_mirror.cluster2.client.mirror.2:Started 2021-03-09T14:33:56.336 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T14:33:59.484 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T14:33:59.493 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T14:33:59.495 INFO:tasks.rbd_mirror_thrash:waiting for 9 secs before reviving daemons 2021-03-09T14:33:59.496 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T14:33:59.488+0000 7fc04ec37700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 90183) UID: 1000 2021-03-09T14:33:59.895 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:34:01.004 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:34:00.998+0000 7f901a51c700 -1 reset not still connected to 0x55890761e5b0 2021-03-09T14:34:08.496 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T14:34:08.503 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:34:08.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:34:08.504 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T14:34:08.504 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T14:34:08.505 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T14:34:08.507 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T14:34:08.508 INFO:tasks.rbd_mirror_thrash:waiting for 69 secs before thrashing 2021-03-09T14:35:03.527 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T14:35:03.523+0000 7f082a074700 -1 reset not still connected to 0x55a99cf08630 2021-03-09T14:35:03.528 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T14:35:03.523+0000 7f082a074700 -1 reset not still connected to 0x55a99f7e6080 2021-03-09T14:35:03.529 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T14:35:03.523+0000 7f082a074700 -1 reset not still connected to 0x55a9acd56b60 2021-03-09T14:35:17.509 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.3412670873310115) > weight (0.25) 2021-03-09T14:35:17.519 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.60082807458382) > weight (0.25) 2021-03-09T14:35:17.519 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.7000680160952234) > weight (0.25) 2021-03-09T14:35:17.519 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.425486669764778) > weight (0.25) 2021-03-09T14:35:17.520 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T14:35:30.338 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.7601212876691508) > weight (0.25) 2021-03-09T14:35:30.354 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.1 2021-03-09T14:35:30.355 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Sent signal 15 2021-03-09T14:35:30.356 INFO:tasks.rbd_mirror_thrash:waiting for 3 secs before reviving daemons 2021-03-09T14:35:30.357 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:2021-03-09T14:35:30.355+0000 7f213b664700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 (PID: 92116) UID: 1000 2021-03-09T14:35:30.557 INFO:tasks.rbd_mirror.cluster2.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:35:33.357 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.1 2021-03-09T14:35:33.377 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:35:33.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:35:33.379 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.1 2021-03-09T14:35:33.379 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Restarting daemon 2021-03-09T14:35:33.382 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.1 2021-03-09T14:35:33.386 INFO:tasks.rbd_mirror.cluster2.client.mirror.1:Started 2021-03-09T14:35:33.387 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T14:36:48.521 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.6587732103791047) > weight (0.25) 2021-03-09T14:36:48.535 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.502448168516981) > weight (0.25) 2021-03-09T14:36:48.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.8344133871892148) > weight (0.25) 2021-03-09T14:36:48.536 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.5980560981015963) > weight (0.25) 2021-03-09T14:36:48.536 INFO:tasks.rbd_mirror_thrash:waiting for 92 secs before thrashing 2021-03-09T14:37:07.388 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T14:37:07.401 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T14:37:07.402 INFO:tasks.rbd_mirror_thrash:waiting for 2 secs before reviving daemons 2021-03-09T14:37:07.403 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T14:37:07.397+0000 7f4ed6cdf700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 91739) UID: 1000 2021-03-09T14:37:08.805 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:37:09.403 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T14:37:09.413 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:37:09.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:37:09.414 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T14:37:09.414 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T14:37:09.415 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T14:37:09.419 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T14:37:09.419 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T14:38:20.537 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.9533749424220869) > weight (0.25) 2021-03-09T14:38:20.548 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.1 2021-03-09T14:38:20.549 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Sent signal 15 2021-03-09T14:38:20.549 INFO:tasks.rbd_mirror_thrash:waiting for 0 secs before reviving daemons 2021-03-09T14:38:20.550 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.1 2021-03-09T14:38:20.550 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:38:20.554 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:2021-03-09T14:38:20.552+0000 7f390f5a2700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 (PID: 91830) UID: 1000 2021-03-09T14:38:20.755 INFO:tasks.rbd_mirror.cluster1.client.mirror.1.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:38:26.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:38:26.653 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.1 2021-03-09T14:38:26.653 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Restarting daemon 2021-03-09T14:38:26.654 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.1 2021-03-09T14:38:26.657 INFO:tasks.rbd_mirror.cluster1.client.mirror.1:Started 2021-03-09T14:38:26.658 INFO:tasks.rbd_mirror_thrash:waiting for 97 secs before thrashing 2021-03-09T14:38:43.420 INFO:tasks.rbd_mirror_thrash:kill cluster2.client.mirror.0 2021-03-09T14:38:43.429 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Sent signal 15 2021-03-09T14:38:43.429 INFO:tasks.rbd_mirror_thrash:waiting for 1 secs before reviving daemons 2021-03-09T14:38:43.431 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:2021-03-09T14:38:43.428+0000 7f0c14d7f700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 (PID: 92593) UID: 1000 2021-03-09T14:38:43.631 INFO:tasks.rbd_mirror.cluster2.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:38:44.430 INFO:tasks.rbd_mirror_thrash:waiting for cluster2.client.mirror.0 2021-03-09T14:38:44.431 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:38:44.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:38:44.432 INFO:tasks.rbd_mirror_thrash:reviving cluster2.client.mirror.0 2021-03-09T14:38:44.432 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Restarting daemon 2021-03-09T14:38:44.432 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster2 --id mirror.0 2021-03-09T14:38:44.435 INFO:tasks.rbd_mirror.cluster2.client.mirror.0:Started 2021-03-09T14:38:44.435 INFO:tasks.rbd_mirror_thrash:waiting for 77 secs before thrashing 2021-03-09T14:40:01.436 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.4191461327371878) > weight (0.25) 2021-03-09T14:40:01.469 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.8388680532050986) > weight (0.25) 2021-03-09T14:40:01.469 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.4422218218776274) > weight (0.25) 2021-03-09T14:40:01.469 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.4446391388503729) > weight (0.25) 2021-03-09T14:40:01.470 INFO:tasks.rbd_mirror_thrash:waiting for 75 secs before thrashing 2021-03-09T14:40:03.659 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.0 with skip (0.34035100082798686) > weight (0.25) 2021-03-09T14:40:03.660 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.1 with skip (0.31336668699088166) > weight (0.25) 2021-03-09T14:40:03.660 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.2 with skip (0.4026457587550921) > weight (0.25) 2021-03-09T14:40:03.660 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster1.client.mirror.3 with skip (0.9895328035152198) > weight (0.25) 2021-03-09T14:40:03.660 INFO:tasks.rbd_mirror_thrash:waiting for 101 secs before thrashing 2021-03-09T14:41:16.470 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.8091156964096706) > weight (0.25) 2021-03-09T14:41:16.480 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.5158912783737718) > weight (0.25) 2021-03-09T14:41:16.481 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.8795100214786435) > weight (0.25) 2021-03-09T14:41:16.481 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.8154510165360053) > weight (0.25) 2021-03-09T14:41:16.481 INFO:tasks.rbd_mirror_thrash:waiting for 94 secs before thrashing 2021-03-09T14:41:44.661 INFO:tasks.rbd_mirror_thrash:kill cluster1.client.mirror.0 2021-03-09T14:41:44.700 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Sent signal 15 2021-03-09T14:41:44.701 INFO:tasks.rbd_mirror_thrash:waiting for 8 secs before reviving daemons 2021-03-09T14:41:44.702 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:2021-03-09T14:41:44.697+0000 7f6e04387700 -1 received signal: Terminated from /usr/bin/python3 /usr/bin/daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 (PID: 92397) UID: 1000 2021-03-09T14:41:44.902 INFO:tasks.rbd_mirror.cluster1.client.mirror.0.smithi144.stderr:daemon-helper: command failed with exit status 1 2021-03-09T14:41:52.701 INFO:tasks.rbd_mirror_thrash:waiting for cluster1.client.mirror.0 2021-03-09T14:41:52.702 INFO:teuthology.orchestra.run:waiting for 600 2021-03-09T14:41:52.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-03-09T14:41:52.703 INFO:tasks.rbd_mirror_thrash:reviving cluster1.client.mirror.0 2021-03-09T14:41:52.703 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Restarting daemon 2021-03-09T14:41:52.704 DEBUG:teuthology.orchestra.run.smithi144:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term rbd-mirror --foreground --cluster cluster1 --id mirror.0 2021-03-09T14:41:52.708 INFO:tasks.rbd_mirror.cluster1.client.mirror.0:Started 2021-03-09T14:41:52.708 INFO:tasks.rbd_mirror_thrash:waiting for 91 secs before thrashing 2021-03-09T14:42:12.080 INFO:tasks.rbd_mirror.cluster1.client.mirror.2.smithi144.stderr:2021-03-09T14:42:12.076+0000 7f7a2a1ac700 -1 rbd::mirror::ImageReplayer: 0x55a1b6e17680 [3/d7f6e04e-71c6-43d7-b9a4-35eacb267eca] start: already running 2021-03-09T14:42:12.081 INFO:tasks.ceph.cluster1.osd.2.smithi092.stderr:2021-03-09T14:42:12.072+0000 7f901a51c700 -1 reset not still connected to 0x558904180b10 2021-03-09T14:42:26.579 INFO:tasks.ceph.cluster2.osd.2.smithi144.stderr:2021-03-09T14:42:26.576+0000 7f082a074700 -1 reset not still connected to 0x55a9994d3400 2021-03-09T14:43:01.118 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.0 with skip (0.5966751889085611) > weight (0.25) 2021-03-09T14:43:01.130 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.1 with skip (0.3485010332867715) > weight (0.25) 2021-03-09T14:43:01.130 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.2 with skip (0.32724124787272435) > weight (0.25) 2021-03-09T14:43:01.130 INFO:tasks.rbd_mirror_thrash:skipping daemon cluster2.client.mirror.3 with skip (0.29487083824503724) > weight (0.25) 2021-03-09T14:43:01.130 INFO:tasks.rbd_mirror_thrash:waiting for 85 secs before thrashing 2021-03-09T14:43:03.930 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-03-09T14:43:03.934 DEBUG:teuthology.task.console_log:Killing console logger for smithi092 2021-03-09T14:43:03.936 DEBUG:teuthology.task.console_log:Killing console logger for smithi144 2021-03-09T14:43:03.936 DEBUG:teuthology.exit:Finished running handlers